Offset decoding device, offset coding device, image filtering device

ABSTRACT

An adaptive offset filter ( 60 ) adds an offset to the pixel value of each pixel forming an input image. The adaptive offset filter ( 60 ) refers to offset-type specifying information, sets offset attributes for a subject unit area of the input image, decodes an offset having a bit width corresponding to an offset value range included in the set offset attributes, and adds the offset to the pixel value of each pixel forming the input image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/820,903, filed on Nov. 22, 2017, which is a continuation of U.S.patent application Ser. No. 15/293,078, filed on Oct. 13, 2016, now U.S.Pat. No. 9,866,833, which is a continuation U.S. patent application Ser.No. 14/127,889, filed on Jan. 31, 2014, now U.S. Pat. No. 9,497,455,which is a National Stage of International Application No.PCT/JP2012/066082, filed on Jun. 22, 2012, which claims the priority ofJapan patent application No. JP2011-139961, filed on Jun. 23, 2011 andJapan Patent Application No. JP2011-215476, filed on Sep. 29, 2011. Allof the afore-mentioned patent applications are hereby incorporated byreference in their entireties.

TECHNICAL FIELD

The present disclosure relates to an image filtering device whichperforms filtering on images. The disclosure also relates to an offsetdecoding device which decodes offsets referred to by an image filter,and an offset coding device which codes offsets referred to by an imagefilter. The disclosure also relates to a data structure of coded data.

BACKGROUND

A video coding device (coding device) which generates coded data bycoding video images in order to transmit or record video imagesefficiently, and a video decoding device (decoding device) whichgenerates decoded images by decoding the coded data are being used.Specific examples of video coding methods are a method defined inH.264/MPEG-4. AVC, a method used in KTA software, which is a jointdevelopment codec in VCEG (Video Coding Expert Group), a method used inTMuC (Test Model under Consideration) software, which is a successorcodec to the codec used in KTA software, and a method used in HM (HEVCTestModel) software.

In such coding methods, images (pictures) forming video images aremanaged in a hierarchical structure which is constituted by slicesobtained by dividing an image, largest coding units (LCU: Largest CodingUnit, also called a tree block) obtained by dividing a slice, codingunits (CU: Coding Unit, also called a coding node) obtained by dividinga largest coding unit, and blocks and partitions obtained by dividing acoding unit. In many cases, images are coded by using blocks as thesmallest coding unit.

Additionally, in such coding methods, normally, a prediction image isgenerated on the basis of a locally decoded image obtained by coding anddecoding an input image, and difference data indicating a differencebetween the prediction image and the input image is coded. As ageneration method for prediction images, inter-frame prediction (interprediction) and intra-frame prediction (intra prediction) are known.

In intra prediction, on the basis of a locally decoded image within thesame frame, prediction images in this frame are sequentially generated.More specifically, in intra prediction, normally, for each unit ofprediction (for example, a block), one of prediction directions(prediction modes) included in a predetermined prediction directiongroup is selected, and also, the pixel value of a reference pixel in alocally decoded image is extrapolated to the selected predictiondirection, thereby generating a prediction pixel value in a subject areato be predicted. On the other hand, in inter prediction, by applyingmotion compensation using motion vectors to a reference image within anentirely decoded reference frame (decoded image), a prediction imagewithin a frame to be predicted is generated for each unit of prediction(for example, a block).

NPL 1 and NPL 2 disclose an adaptive offset filter disposed at a stagesubsequent to a deblocking filter which reduces block distortion of adecoded image and at a stage prior to an adaptive loop filter (alsoreferred to as an “adaptive filer”) which performs filtering processingusing an adaptively determined filter coefficient. This adaptive offsetfilter adds an adaptively set offset to the pixel value of each pixel ofan image output from the deblocking filter.

By providing such an adaptive offset filter, it is possible to suppressblock distortion more effectively.

CITATION LIST Non Patent Literature

-   NPL 1: “JCTVC-D122”, Joint Collaborative Team on Video Coding    (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 4th Meeting:    Daegu, KR, 01/2011-   NPL 2: “JCTVC-E049”, Joint Collaborative Team on Video Coding    (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 5th Meeting:    Geneva, CH, 03/2011

SUMMARY OF INVENTION Technical Problem

However, the range of values of an offset used in a known adaptiveoffset filter is not set, and thus, the number of bits of an offset islarge, which requires a large memory size for storing an offset.

The present disclosure has been made in view of the above-describedproblem. It is an object of the present invention to realize an imagefiltering device which is capable of reducing block distortion whilesuppressing an increase in the memory size.

In order to solve the above-described problem, an image filtering deviceaccording to the present disclosure is an image filtering device foradding an offset to a pixel value of each pixel forming an input imagewhich is constituted by a plurality of unit areas. The image filteringdevice includes: offset attribute setting means for setting an offsetvalue range by referring to coded data; offset decoding means fordecoding an offset which is restricted to the set offset value range;and filtering means for adding the offset to the pixel value of eachpixel forming the input image.

With the image filtering device configured as described above, theoffset attribute setting means sets an offset value range, and theoffset decoding means decodes an offset having a bit width correspondingto an offset value range included in the set offset value range. It isthus possible to effectively reduce the memory size of a memory forstoring offsets.

Accordingly, with the above-described configuration, it is possible toperform appropriate offset filtering processing while the memory size ofa memory for storing offsets is reduced.

An offset decoding device according to the present invention is anoffset decoding device for decoding each offset which is referred to byan image filter for adding an offset to a pixel value of each pixelforming an input image. The offset decoding device includes: offsetresidual decoding means for decoding each offset residual from codeddata; prediction value determining means for determining a predictionvalue of each offset from a decoded offset; and offset calculating meansfor calculating each offset from a prediction value determined by theprediction value determining means and an offset residual decoded by theoffset residual decoding means.

In the offset decoding device configured as described above, there areprovided the offset residual decoding means for decoding each offsetresidual from coded data, the prediction value determining means fordetermining a prediction value of each offset from a decoded offset, andthe offset calculating means for calculating each offset from aprediction value determined by the prediction value determining meansand an offset residual decoded by the offset residual decoding means.Accordingly, an offset can be appropriately decoded from coded datahaving a smaller amount of data, compared with a case in which eachoffset itself is coded.

An image filtering device according to the present invention is an imagefiltering device which operates on an input image. The image filteringdevice includes: calculating means for calculating a difference valuebetween a pixel value of a subject pixel forming an input image and apixel value of a pixel around the subject pixel; bit shift means forperforming bitwise right shift on a pixel value referred to by thecalculating means or the difference value calculated by the calculatingmeans by an amount equal to a predetermined shift value; classifyingmeans for classifying the subject pixel as one of a plurality of offsetclasses in accordance with a magnitude relation between the differencevalue subjected to bitwise right shift by the bit shift means and 0; andoffset means for adding an offset associated with the offset class ofthe subject pixel classified by the classifying means to the pixel valueof the subject pixel.

In the image filtering device configured as described above, the subjectpixel is classified as one of a plurality of offset classes inaccordance with a magnitude relation between the difference valuesubjected to bitwise right shift by the bit shift means and 0, and anoffset associated with the offset class of the subject pixel classifiedby the classifying means is added to the pixel value of the subjectpixel. Thus, the classifying processing is less vulnerable to theinfluence of noise, thereby making it possible to improve the codingefficiency.

An image filtering device according to the present invention is an imagefiltering device which operates on an input image. The image filteringdevice includes: calculating means for calculating a difference valuebetween a pixel value of a subject pixel forming an input image and apixel value of a pixel around the subject pixel; classifying means forclassifying the subject pixel as one of a plurality of offset classes inaccordance with a magnitude relation between the difference valuecalculated by the calculating means and each of predetermined first andsecond thresholds; and offset means for adding an offset associated withthe offset class of the subject pixel classified by the classifyingmeans to the pixel value of the subject pixel.

In the image filtering device configured as described above, the subjectpixel is classified as one of a plurality of offset classes inaccordance with a magnitude relation between the difference valuecalculated by the calculating means and each of the predetermined firstand second thresholds, and an offset associated with the offset class ofthe subject pixel classified by the classifying means is added to thepixel value of the subject pixel. Thus, the classifying processing isless vulnerable to the influence of noise, thereby making it possible toimprove the coding efficiency.

An image filtering device according to the present invention is an imagefiltering device which operates on an input image constituted by aplurality of unit areas. The image filtering device includes:determining means for determining, among first and second offset types,an offset type to which a subject unit area including a subject pixelforming the input image belongs; classifying means for classifying thesubject pixel as one of an offset class in which an offset is not addedand a plurality of offset classes in which an offset is added inaccordance with the offset type to which the subject unit area belongsand a pixel value of the subject pixel; and offset means for adding anoffset associated with the offset type to which the subject unit areabelongs and the offset class of the subject pixel classified by theclassifying means to the pixel value of the subject pixel. In a case inwhich the pixel value of the subject pixel is within a predeterminedrange, the classifying means classifies the subject pixel as an offsetclass in which an offset is added, regardless of whether the offset typeto which the unit area including the subject pixel belongs is the firstoffset type or the second offset type.

In the image filtering device configured as described above, in a casein which the pixel value of the subject pixel is within a predeterminedrange, the subject pixel is classified as an offset class in which anoffset is added, regardless of whether the offset type to which the unitarea including the subject pixel belongs is the first offset type or thesecond offset type, thereby making it possible to effectively eliminateblock noise. Accordingly, with the above-described configuration, thecoding efficiency can be improved.

An image filtering device according to the present invention is an imagefiltering device for adding an offset to a pixel value of each pixelforming an input image which is constituted by a plurality of unitareas. The image filtering device includes: determining means fordetermining an offset type to which a subject unit area belongs among aplurality of offset types; offset coding means for determining an offsethaving a bit width which differs depending on the offset type and forcoding the offset; and filtering means for adding the determined offsetto the pixel value of each pixel forming the input image.

In the image filtering device configured as described above, among aplurality of offset types, the offset type to which a subject unit areabelongs is determined, an offset having a bit width which differsdepending on the determined offset type is determined, and thedetermined offset is added to the pixel value of each pixel forming theinput image. The determined offset is also coded.

Accordingly, with the above-described configuration, it is possible toperform appropriate offset filtering processing while the memory size ofa memory for storing offsets is reduced. With the above-describedconfiguration, since the amount of data required to code data isreduced, the coding efficiency is improved.

An offset coding device according to the present invention is an offsetcoding device for coding each offset which is referred to by an imagefilter for adding an offset to a pixel value of each pixel forming aninput image. The offset coding device includes: prediction valuedetermining means for determining a prediction value of each offset froma coded offset; offset residual calculating means for calculating anoffset residual from each offset and a prediction value determined bythe prediction value determining means; and offset residual coding meansfor coding an offset residual calculated by the offset residualcalculating means.

In the offset coding device configured as described above, there areprovided the prediction value determining means for determining aprediction value of each offset from a coded offset, the offset residualcalculating means for calculating an offset residual from each offsetand a prediction value determined by the prediction value determiningmeans, and the offset residual coding means for coding an offsetresidual calculated by the offset residual calculating means. It is thuspossible to reduce the amount of data required to code data can bereduced.

A data structure of coded data according to the present invention is adata structure of coded data which is referred to by an image filter foradding an offset to a pixel value of each pixel forming an input imagewhich is constituted by a plurality of unit areas. The data structureincludes: offset-type specifying information which specifies an offsettype to which each unit area belongs; and an offset having a bit widthwhich differs depending on the offset type. The image filter refers tothe offset-type specifying information included in the coded data, anddetermines an offset type to which a subject unit area belongs and alsodecodes an offset having a bit width which differs depending on thedetermined offset type.

The coded data configured as describe above includes an offset having abit width which differs depending on the offset type, thereby reducingthe amount of data required to code data. The image filter which decodesthe coded data refers to the offset-type specifying information, anddetermines the offset type to which a subject unit area belongs and alsodecodes an offset having a bit width which differs depending on thedetermined offset type. It is thus possible to perform appropriateoffset filtering processing while the memory size of a memory forstoring offsets is reduced.

The offset-type specifying information may be determined for each of theinput images or for each of the unit areas. Alternatively, theoffset-type specifying information may be determined for eachpredetermined set of the input images or for each predetermined set ofthe unit areas.

Advantageous Effects of Invention

As described above, an image filtering device according to the presentinvention is an image filtering device for adding an offset to a pixelvalue of each pixel forming an input image which is constituted by aplurality of unit areas. The image filtering device includes: offsetattribute setting means for setting an offset value range by referringto coded data; offset decoding means for decoding an offset which isrestricted to the set offset value range; and filtering means for addingthe offset to the pixel value of each pixel forming the input image.

An image filtering device according to the present invention is an imagefiltering device for adding an offset to a pixel value of each pixelforming an input image which is constituted by a plurality of unitareas. The image filtering device includes: determining means fordetermining an offset type to which a subject unit area belongs among aplurality of offset types; offset coding means for determining an offsethaving a bit width which differs depending on the offset type and forcoding the offset; and filtering means for adding the determined offsetto the pixel value of each pixel forming the input image.

A data structure of coded data according to the present invention is adata structure of coded data which is referred to by an image filter foradding an offset to a pixel value of each pixel forming an input imagewhich is constituted by a plurality of unit areas. The data structureincludes: offset-type specifying information which specifies an offsettype to which each unit area belongs; and an offset having a bit widthwhich differs depending on the offset type. The image filter refers tothe offset-type specifying information included in the coded data, anddetermines an offset type to which a subject unit area belongs and alsodecodes an offset having a bit width which differs depending on thedetermined offset type.

With the above-described configuration, it is possible to cause an imagefilter to perform appropriate offset filtering processing while thememory size of a memory for storing offsets is reduced.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating the configuration of an adaptiveoffset filter according to a first embodiment of the present invention.

FIG. 2 illustrates a data structure of coded data generated by a videocoding device according to the first embodiment of the present inventionand decoded by a video decoding device according to the first embodimentof the present invention: parts (a) through (d) respectively illustratea picture layer, a slice layer, a tree block layer, and a CU layer, part(e) illustrates the configuration of QAOU information concerning a QAOUwhich is not a leaf; and part (f) illustrates the configuration of QAOUinformation concerning a QAOU which is a leaf.

FIG. 3 illustrates each syntax included in offset information OIconcerning coded data according to the first embodiment of the presentinvention.

FIG. 4 shows split modes of offset units according to the firstembodiment of the present invention: part (a) shows a split mode whensao_curr_depth=0; part (b) shows a split mode when sao_curr_depth=1;part (c) shows a split mode when sao_curr_depth=2; part (d) shows asplit mode when sao_curr_depth=3; and part (e) shows a split mode whensao_curr_depth=4.

FIG. 5 is a block diagram illustrating the configuration of the videodecoding device according to the first embodiment of the presentinvention.

FIG. 6 illustrates the first embodiment of the present invention: part(a) shows QAOMUs having a split depth of three and forming a subjectunit of processing and QAOU indexes assigned to QAOMUs; and part (b)shows offset types associated with QAOU indexes 0 to 9 and offsets withrespect to individual classes that can be selected for each offset type.

FIG. 7 illustrates examples of QAOMU numbers appended to QAOMUs includedin a subject unit of processing: part (a) shows a QAOMU number assignedto a QAOMU having a split depth of 0; part (b) shows QAOMU numbersassigned to QAOMUs having a split depth of one; part (c) shows QAOMUnumbers assigned to QAOMUs having a split depth of two; part (d) showsQAOMU numbers assigned to QAOMUs having a split depth of three; and part(e) shows QAOMU numbers assigned to QAOMUs having a split depth of four.

FIG. 8 is a flowchart of a flow of processing performed by an adaptiveoffset filter processor according to the first embodiment of the presentinvention.

FIG. 9 illustrates, together with pixel bit depths, examples of offsetbit depths and shift values set by an offset attribute setting sectionincluded in an adaptive filter according to the first embodiment of thepresent invention: parts (a) through (d) respectively show examplesassociated with patterns S1 through S4; and part (e) shows a case inwhich STEP=2 in part (d).

FIG. 10 illustrates offset processing performed by the adaptive offsetfilter according to the first embodiment of the present invention: parts(a) through (d) show pixels to be referred to when sao_type_idx=1 to 4,respectively.

FIG. 11 illustrates offset processing performed by the adaptive offsetfilter according to the first embodiment of the present invention: part(a) shows graphs indicating the magnitude relations between a pixelvalue pic[x] of a subject pixel x and the pixel value of a pixel a or band also shows the values of a function Sign in accordance with themagnitude relation; part (b) shows graphs indicating the magnituderelations between the pixel value of the subject pixel x and each of thepixel values of the pixel a and the pixel b and also shows the values ofEdgeType in accordance with the magnitude relation; part (c) shows theassociation between each graph shown in part (b) and class_idx; and part(d) illustrates a transform table indicating transformation fromEdgeType to class_idx.

FIG. 12 illustrates offset processing performed by the adaptive offsetfilter according to the first embodiment of the present invention: part(a) schematically shows classifying performed when sao_type_idx=5; part(b) schematically shows classifying performed when sao_type_idx=6; andpart (c) shows a table indicating an example of classifying performedwhen a band offset is specified.

FIG. 13 illustrates offset processing performed by the adaptive offsetfilter according to the first embodiment of the present invention andshows a table indicating another example of classifying performed when aband offset is specified.

FIG. 14 is a block diagram illustrating the configuration of a videocoding device according to the first embodiment of the presentinvention.

FIG. 15 is a block diagram illustrating the configuration of an adaptiveoffset filter included in the video coding device according to the firstembodiment of the present invention.

FIG. 16 is a flowchart of a flow of processing performed by an offsetcalculator of the adaptive offset filter included in the video codingdevice according to the first embodiment of the present invention.

FIG. 17 is a flowchart of a flow of processing performed by an offsetinformation selector of the adaptive offset filter included in the videocoding device according to the first embodiment of the presentinvention.

FIG. 18 illustrates processing performed by the offset informationselector of the adaptive offset filter included in the video codingdevice according to the first embodiment of the present invention: part(a) shows a split mode when the split depth is 0 and 1; part (b) shows asplit mode when the split depth is 1; part (c) shows a split mode whenthe split depth is 2; and part (d) shows an example of split modedetermined by the offset information selector.

FIG. 19 is a block diagram illustrating the configuration of an adaptiveoffset filter included in a video decoding device according to a secondembodiment of the present invention.

FIG. 20 illustrates the adaptive offset filter according to the secondembodiment of the present invention: part (a) illustrates a firstspecific example of a function merge_tbl[sao_type_idx]; and part (b)illustrates a second specific example of the functionmerge_tbl[sao_type_idx].

FIG. 21 is a block diagram illustrating the configuration of an adaptiveoffset filter included in a video coding device according to the secondembodiment of the present invention.

FIG. 22 is a block diagram illustrating the configuration of an adaptiveoffset filter according to a third embodiment of the present invention.

FIG. 23 illustrates syntax of offset information OI of coded dataaccording to the third embodiment.

FIG. 24 illustrates the third embodiment: part (a) shows QAOMUs having asplit depth of three and forming a subject unit of processing and alsoshows QAOU indexes assigned to QAOMUs; and part (b) shows an offset typeassociated with each of the QAOU indexes 0 to 9 and offsets with respectto individual classes that can be selected for each offset type.

FIG. 25 illustrates transform tables used by the offset informationdecoding section according to the third embodiment.

FIG. 26 illustrates offset processing performed by the adaptive offsetfilter according to the third embodiment: part (a) shows graphsindicating the magnitude relations between the pixel value pic[x] of asubject pixel x and the pixel value of a pixel a or b and also shows thevalues of a function Sign in accordance with the magnitude relation;part (b) shows graphs indicating the magnitude relations between thepixel value of the subject pixel x and each of the pixel values of thepixel a and the pixel b and also shows the value of EdgeType inaccordance with the magnitude relation; part (c) shows the associationbetween each graph shown in part (b) and class_idx; and parts (d)through (f) illustrate transform tables used for transforming fromEdgeType to class_idx.

FIG. 27 illustrates offset processing performed by the adaptive offsetfilter according to the third embodiment: part (a) schematically showsclassifying performed when “sao_type_idx=”5; and part (b) schematicallyshows classifying performed when “sao_type_idx=6”.

FIG. 28 illustrates offset processing performed by the adaptive offsetfilter according to the third embodiment: part (a) schematically showsclassifying performed when the hierarchical depth of a subject QAOU issmaller than a threshold; and part (b) schematically shows classifyingperformed when the hierarchical depth of a subject QAOU is equal to orgreater than the threshold.

FIG. 29 illustrates an example of classifying performed when a bandoffset is specified: part (a) shows an example of classifying performedwhen the hierarchical depth of a subject QAOU is smaller than athreshold; and part (b) shows an example of classifying performed whenthe hierarchical depth of a subject QAOU is equal to or greater than thethreshold.

FIG. 30 is a block diagram illustrating the configuration of an adaptiveoffset filter 80 included in a video coding device according to thethird embodiment.

FIG. 31 shows an overview of a case in which a squared error concerninga QAOU having a QAOU index “x” is calculated for each offset type in thethird embodiment.

FIG. 32 illustrates the configuration in which EO and BO are switched inaccordance with the pixel value in a fourth embodiment of the presentinvention: part (a) shows an overview of the configuration in which EOand BO are switched in accordance with the pixel value; part (b) showsspecific values to be switched; and part (c) shows the content of a listmemory stored in the offset information storage section 621.

FIG. 33 illustrates a case in which the EO type is restricted to thehorizontal direction in a fifth embodiment of the present invention:part (a) shows an overview of advantages; part (b) shows the positionsof pixels in the horizontal direction; part (c) shows a state differentfrom the state in part (b); and part (d) shows an overview of advantagesobtained by the state in part (c).

FIG. 34 illustrates a case in which the EO type is restricted to thehorizontal direction in the fifth embodiment of the present invention:parts (a) and (b) show cases in which reference pixels are positionedasymmetrically; and part (c) shows an overview of horizontal edges.

FIG. 35 shows an overview of a case in which the offset precision isimproved in a sixth embodiment of the present invention.

FIG. 36 illustrates an overview of a case in which more detailedclassifying is performed in the sixth embodiment: part (a) shows atransform table; and parts (b) through (d) show classifying.

FIG. 37 illustrates a case in which classifying is performed inaccordance with the chrominance in the sixth embodiment; part (a) showsclassifying performed by considering the pixel value of an achromaticcolor, part (b) shows classifying in which two value ranges areasymmetrically positioned with respect to the pixel values of anachromatic color, parts (c) and (d) show a case in which classifying isperformed differently in accordance with the chrominance channel (Cr orCb); and part (e) shows a case in which classifying is performed byusing only one BO type.

FIG. 38 is a block diagram illustrating the configuration of an offsetinformation decoding section according to a seventh embodiment of thepresent invention.

FIG. 39 illustrates an overview of a case in which there are classesunder which no pixels are classified in the seventh embodiment.

FIG. 40 illustrates syntax to be used when a prediction candidate flagis used in the seventh embodiment.

FIG. 41 is a block diagram illustrating the configuration of an offsetinformation decoding section according to the fifth embodiment.

FIG. 42 Part (a) of FIG. 42 is a block diagram illustrating theconfiguration of an offset type selector according to the fifthembodiment; and part (b) is a block diagram illustrating theconfiguration of another offset type selector.

FIG. 43 is a block diagram illustrating the configuration of aclassifying section according to the fifth embodiment.

FIG. 44 illustrates syntax of offset information and syntax of QAOUinformation according to the third embodiment: part (a) illustratessyntax of offset information; part (b) illustrates syntax of QAOUinformation; and part (c) illustrates the entire syntax of an adaptiveoffset filter which calls syntax of part (a) and syntax of part (b).

FIG. 45 illustrates a case in which a video decoding device and a videocoding device can be used for transmitting and receiving video images;part (a) is a block diagram illustrating the configuration of atransmitting apparatus on which the video coding device is mounted; andpart (b) is a block diagram illustrating the configuration of areceiving apparatus on which the video decoding device is mounted.

FIG. 46 illustrates a case in which a video decoding device and a videocoding device can be used for recording and playing back video images;part (a) is a block diagram illustrating the configuration of arecording apparatus on which the video coding device 2 is mounted; andpart (b) is a block diagram illustrating the configuration of a playbackapparatus on which the video decoding device is mounted.

DESCRIPTION OF EMBODIMENTS First Embodiment

(Coded Data #1)

Prior to a detailed description of a video coding device 2 and a videodecoding device 1 according to a first embodiment, the data structure ofcoded data #1 to be generated by the video coding device 2 and to bedecoded by the video decoding device 1 will be discussed below.

FIG. 2 illustrates the data structure of the coded data #1. The codeddata #1 includes a sequence and a plurality of pictures forming thesequence by way of example.

The structures of hierarchical levels of a picture layer or lower layersof the coded data #1 are shown in FIG. 2. Parts (a) through (d) of FIG.2 respectively illustrate a picture layer which defines a picture PICT,a slice layer which defines a slice S, a tree block layer which definesa tree block (Tree block) TBLK, and a CU layer which defines a codingunit (Coding Unit; CU) included in the tree block TBLK.

(Picture Layer)

In the picture layer, a set of data items to be referred to by the videodecoding device 1 in order to decode a picture PICT to be processed(hereinafter also referred to as a “subject picture”) are defined. Thepicture PICT includes, as shown in part (a) of FIG. 2, a picture headerPH and slices S₁ through S_(NS) (NS is the total number of slicesincluded in the picture PICT).

If it is not necessary to distinguish the individual slices S₁ throughS_(NS) from each other, the subscripts of the reference numerals may beomitted. Other items of data provided with subscripts included in thecoded data #1 described below are handled in a similar manner.

The picture header PH includes a coding parameter set to be referred toby the video decoding device 1 in order to determine a decoding methodfor a subject picture. For example, coding mode information(entropy_coding_mode_flag) which indicates the variable-length codingmode used when the video coding device 2 has performed coding is anexample of coding parameters included in the picture header PH.

If entropy_coding_mode_flag is 0, the picture PICT has been coded byCAVLC (Context-based Adaptive Variable Length Coding). Ifentropy_coding_mode_flag is 1, the picture PICT has been coded by CABAC(Context-based Adaptive Binary Arithmetic Coding).

The picture header PH is also referred to as a “picture parameter set(PPS: Picture Parameter Set).

(Slice Layer)

In the slice layer, a set of data items to be referred to by the videodecoding device 1 in order to decode a slice S to be processed(hereinafter also referred to as a “subject slice”) are defined. Theslice S includes, as shown in part (b) of FIG. 2, a slice header SH anda sequence of tree blocks TBLK₁ through TBLK_(NC) (NC is the totalnumber of tree blocks included in the slice S).

The slice header SH includes a coding parameter set to be referred to bythe video decoding device 1 in order to determine a decoding method fora subject slice. Slice-type specifying information (slice_type) whichspecifies a slice type is an example of coding parameters included inthe slice header SH.

Examples of slice types that can be specified by the slice-typespecifying information are (1) I slice which uses only intra predictionwhen coding is performed, (2) P slice which uses unidirectionalprediction or intra prediction when coding is performed, and (3) B slicewhich uses unidirectional prediction, bidirectional prediction, or intraprediction when coding is performed.

The slice header SH also includes a filter parameter FP to be referredto by an adaptive filter provided in the video decoding device 1. Thefilter parameter FP may be included in the picture header PH.

(Tree Block Layer)

In the tree block layer, a set of data items to be referred to by thevideo decoding device 1 in order to decode a tree block TBLK to beprocessed (hereinafter also referred to as a “subject tree block”) aredefined. The tree block may also be referred to as a largest coding unit(LCU: Largest Coding Unit).

The tree block TBLK includes a tree block header TBLKH and items ofcoding unit information CU₁ through CU_(NL) (NL is the total number ofitems of coding unit information included in the tree block TBLK). Therelationship between the tree block TBLK and the coding unit informationCU will first be discussed below.

The tree block TBLK is split into partitions for specifying the blocksize which is used for each of processing operations, such as intraprediction, inter prediction, and transform.

The partitions of the tree block TBLK are obtained by recursive quadtreesplitting. Hereinafter, the tree structure obtained by this recursivequadtree splitting will be referred to as a “coding tree”.

Hereinafter, a partition corresponding to a leaf, which is an end nodeof a coding tree, will be referred to as a coding node. The coding nodeis a basic unit of coding processing, and thus, it will also be referredto as a “coding unit (CU)”.

That is, each of the items of coding unit information CU₁ throughCU_(NL) (hereinafter referred to as “CU information”) is informationcorresponding to each coding node (coding unit) obtained by performingrecursive quadtree splitting on the tree block TBLK.

The root of the coding tree is associated with the tree block TBLK. Inother words, the tree block TBLK is associated with the highest node ofa quadtree splitting tree structure in which a plurality of coding nodesare included in a recursive manner.

The vertical and horizontal sizes of each coding node are half thevertical and horizontal sizes of a higher coding node to which thecoding node directly belongs (that is, a partition of a node one levelhigher than the coding node).

The possible sizes of each coding node are dependent on size specifyinginformation and maximum hierarchical depth of the coding node includedin a sequence parameter set SPS of the coded data #1. For example, ifthe size of a tree block TBLK is 64×64 pixels and the maximumhierarchical depth is three, each of the coding nodes of the tree blockTBLK and lower levels may take one of the three sizes, that is, 64×64pixels, 32×32 pixels, and 16×16 pixels.

(Tree Block Header)

In the tree block header TBLKH, coding parameters referred to by thevideo decoding device 1 in order to determine a decoding method for asubject tree block are included. More specifically, as shown in part (c)of FIG. 2, tree block split information SP_TBLK which specifies a splitpattern for splitting a subject tree block into CUs and a quantizationparameter difference Δqp (qp_delta) which specifies the quantizationstep size are included.

The tree block split information SP_TBLK is information indicating acoding tree for splitting a tree block, and more specifically, theconfiguration and the size of each of the CUs included in the subjecttree block and the position of each CU within the subject tree block.

The tree block split information SP_TBLK does not have to explicitlyinclude the configuration and the size of each CU. For example, the treeblock split information SP_TBLK may be a set of flags(split_coding_unit_flag) indicating whether or not the entre subjecttree block or sub-regions of the tree block will be subjected toquadtree splitting. In this case, the configuration and the size of thetree block are utilized together with the set of flags, thereby makingit possible to specify the configuration and the size of each CU.

The quantization parameter difference Δqp is a difference qp-qp′ betweenthe quantization parameter qp used in the subject tree block and thequantization parameter qp′ used in a tree block which has been codedimmediately before the subject tree block.

(CU Layer)

In the CU layer, a set of data items referred to by the video decodingdevice 1 for decoding a CU to be processed (hereinafter also referred toas a “subject CU”) are defined.

Prior to a specific description of data items included in CU informationCU, the tree structure of data items included in a CU will be discussedbelow. The coding node is a node of a root of a prediction tree (PT) ora node of a root of a transform tree (TT). The prediction tree and thetransform tree will be discussed as follows.

In the prediction tree, a coding node is split into one or a pluralityof prediction blocks, and the position and the size of each predictionblock are defined. In other words, one prediction block or theprediction blocks are one or a plurality of non-overlapping regionsforming a coding node. The prediction tree includes one or a pluralityof prediction blocks obtained by the above-described splitting.

Prediction processing is performed on each prediction block. Theprediction block, which is a unit of prediction, will also be referredto as a “prediction unit (PU)”.

Broadly speaking, split methods used in a prediction tree are dividedinto two types, that is, one type using intra prediction and the othertype using inter prediction.

In the type of intra prediction, the split method includes splitpatterns of 2N×2N (the same size as that of a coding node) and N×N.

In the type of inter prediction, the split method includes splitpatterns of 2N×2N (the same size as that of the coding node), 2N×N,N×2N, and N×N.

In the transform tree, a coding node is split into one or a plurality oftransform blocks, and the position and the size of each transform blockare defined. In other words, one transform block or the transform blocksare one or a plurality of non-overlapping regions forming a coding node.The transform tree includes one or a plurality of transform blocksobtained by the above-described splitting.

Split methods used in the transform tree are divided into the followingtypes: assigning a region having the same size as that of a coding nodeas the transform block; and assigning regions obtained by performingrecursive quadtree splitting, as in the splitting of the above-describedtree block, as transform blocks.

Transform processing is performed on each transform block. The transformblock, which is a unit of transform, will also be referred to as a“transform unit (TU)”.

(Data Structure of CU Information)

Data included in the CU information CU will now be specificallydiscussed below with reference to part (d) of FIG. 2. Specifically, theCU information CU includes, as shown in part (d) of FIG. 2, a skip flagSKIP, PT information PTI, and TT information TTI.

The skip flag SKIP is a flag indicating whether or not a skip mode isapplied to a subject PU. If the value of the skip flag SKIP is 1, thatis, if the skip mode is applied to a subject CU, the PT information PTIand the TT information TTI of this CU information CU are omitted. Theskip flag SKIP is omitted in I slices.

The PT information PTI is information concerning a PT included in theCU. In other words, the PT information PTI is a set of information itemsconcerning one or a plurality of PUs included in the PT and is referredto by the video decoding device 1 when generating prediction images. ThePT information PTI includes, as shown in part (d) of FIG. 2, predictiontype information PType and prediction information PInfo.

The prediction type information PType is information which specifieswhether intra prediction or inter prediction will be used as theprediction-image generating method for the subject PU.

The prediction information PInfo is constituted by intra-predictioninformation or inter-prediction information, depending on whether theprediction type information PType specifies intra prediction or interprediction. Hereinafter, a PU to which intra prediction is applied mayalso be referred to as an “intra PU”, and a PU to which inter predictionis applied may also be referred to as an “inter PU”.

The prediction information PInfo also includes information whichspecifies the configuration, size, and position of the subject PU. Asstated above, the generation of a prediction image is performed by usinga PU as a unit. Details of the prediction information PInfo will bediscussed later.

The TT information TTI is information concerning a TT included in theCU. In other words, the TT information TTI is a set of information itemsconcerning one or a plurality of TUs included in the TT and is referredto by the video decoding device 1 when decoding residual data.Hereinafter, a TU may also be referred to as a “block”.

The TT information TTI includes, as shown in part (d) of FIG. 2, TTsplit information SP_TT which specifies a split pattern used forsplitting the subject CU into transform blocks and quantized predictionresiduals QD₁ through QD_(NT) (NT is the total number of blocks includedin the subject CU).

The TT split information SP_TT is information for determining theconfiguration and the size of each TU included in the subject CU and theposition of each TU within the subject CU. For example, the TT splitinformation SP_TT may be implemented by information(split_transform_unit_flag) indicating whether or not a subject nodewill be split and information (trafoDepth) indicating the depth of thesplitting.

If the size of a CU is 64×64, the size of each of TUs obtained bysplitting the CU may be in a range of 32×32 pixels to 2×2 pixels.

Each quantized prediction residual QD is coded data generated as aresult of the video coding device 2 executing the following Processing 1through Processing 3 on a subject block, which is a block to beprocessed.

Processing 1: performing DCT transform (Discrete Cosine Transform) on aprediction residual obtained by subtracting a prediction image from animage to be coded;

Processing 2: quantizing a transform coefficient obtained in Processing1; and

Processing 3: performing variable-length coding on the transformcoefficient quantized in Processing 2.

The above-described quantization parameter qp represents a quantizationstep size QP (QP=2^(qp/6)) used when the video coding device 2 quantizesthe transform coefficient.

(Prediction Information PInfo)

As stated above, the prediction information PInfo has two types, thatis, inter-prediction information and intra-prediction information.

The inter-prediction information includes coding parameters referred toby the video decoding device 1 when generating an inter-prediction imageby using inter prediction. More specifically, the inter-predictioninformation includes inter PU split information which specifies a splitpattern used for splitting a subject CU into inter PUs andinter-prediction parameters concerning individual inter PUs.

The inter-prediction parameters include a reference image index, anestimation motion-vector index, and a motion-vector residual.

The intra-prediction information includes coding parameters referred toby the video decoding device 1 when generating an intra-prediction imageby using intra prediction. More specifically, the intra-predictioninformation includes intra PU split information which specifies a splitpattern used for splitting a subject CU into intra PUs andintra-prediction parameters concerning individual intra PUs. Theintra-prediction parameter is a parameter for specifying anintra-prediction method (prediction mode) for each intra PU.

(Offset Unit)

In this embodiment, each picture or each slice is recursively split intoa plurality of offset units (may also be referred to as “QAOU: QuadAdaptive Offset Unit”) in accordance with a quadtree structure. The QAOUis a processing unit of offset filtering processing performed by anadaptive offset filter of this embodiment.

As shown in parts (e) and (f) of FIG. 2, QAOU information, which isinformation concerning each QAOU, includes sao_split_flag indicatingwhether or not the QAOU will be further split. More specifically,sao_split_flag is specified by arguments (sao_curr_depth, ys, xs), whichwill be discussed later, and is also represented bysao_split_flag[sao_curr_depth][ys][xs].

If sao_split_flag included in a QAOU indicates that the QAOU will befurther split (that is, if the QAOU is not a leaf), the QAOU informationconcerning the QAOU includes, as shown in part (e) of FIG. 2, items ofQAOU information concerning a plurality of QAOUs included in the QAOU.

On the other hand, if sao_split_flag included in a QAOU indicates thatthe QAOU will not be further split (that is, if the QAOU is a leaf), theQAOU information concerning the QAOU includes, as shown in part (f) ofFIG. 2, offset information OI concerning the QAOU. The offsetinformation OI includes, as shown in part (f) of FIG. 2, offset-typespecifying information OTI which specifies an offset type and an offsetgroup which is determined in accordance with the offset type. As shownin part (f) of FIG. 2, the offset group includes a plurality of offsets.

An offset included in coded data is a quantized value. An offsetincluded in coded data may be a prediction residual obtained by using acertain prediction, for example, a linear prediction. If the pixel bitdepth, which will be discussed later, is variably set for each unitarea, the pixel bit depth for an area to be processed may be included inthe offset-type specifying information OTI, since the offset bit depth,the offset value range, and the shift value are changed in accordancewith the pixel bit depth in this embodiment.

The offset information OI will be discussed below by changing drawingsfor reference.

FIG. 3 illustrates each syntax included in the offset information OI(indicated by sao_offset_param( ) in FIG. 3).

As shown in FIG. 3, the offset information OI includes syntaxsao_type_idx[sao_curr_depth][ys][xs]. Ifsao_type_idx[sao_curr_depth][ys][xs] is not 0,sao_offset[sao_curr_depth][ys][xs][i] is included in the offsetinformation OI.

(sao_curr_depth, ys, xs)

An argument sao_curr_depth, which is an argument of sao_type idx andsao_offset, is an index indicating the split depth of a QAOU, and ys andxs are indexes respectively indicating the position in the y directionand the position in the x direction of a QAOU (or a QAOMU, which will bediscussed later).

Parts (a) through (e) of FIG. 4 show split modes of a QAOU in accordancewith the values of sao_curr_depth: part (a) shows a split mode whensao_curr_depth=0, part (b) shows a split mode when sao_curr_depth=1,part (c) shows a split mode when sao_curr_depth=2, part (d) shows asplit mode when sao_curr_depth=3, and part (e) shows a split mode whensao_curr_depth=4. As shown in parts (a) through (e) of FIG. 4, each QAOUis specified by sao_curr_depth and (xs, ys).

As shown in part (a) of FIG. 4, when sao_curr_depth=0, xs and ys areboth 0. As shown in part (b) of FIG. 4, when sao_curr_depth=1, xs and ystake 0 or 1. As shown in part (c) of FIG. 4, when sao_curr_depth=2, xsand ys take one of 0, 1, 2, and 3. Generally, in response to a givenvalue of sao_curr_depth, xs and ys take values of 0 to2^(sao_curr_depth)−1.

(sao_type_idx)

Syntax sao_type_idx[sao_curr_depth][ys][xs] corresponds to theabove-described offset-type specifying information OTI, and is syntaxfor specifying the offset type of each QAOU. Hereinafter, sao_type_idxmay be simply referred to as an “offset type”.

In this embodiment, sao_type_idx[sao_curr_depth][ys][xs] takes integersfrom 0 to 6. When sao_type_idx[sao_curr_depth][ys][xs]=0, it indicatesthat offset filtering processing is not performed on an image which hasnot been yet subjected to offset filtering (for example, a deblockeddecoded image P_DB, which will be discussed later) in a subject QAOU.When sao_type_idx[sao_curr_depth][ys][xs]=1 to 4, it indicates that edgeoffset processing is performed on an image which has not been yetsubjected to offset filtering in a subject QAOU. Whensao_type_idx[sao_curr_depth][ys][xs]=5 or 6, it indicates that bandoffset processing is performed on an image which has not been yetsubjected to offset filtering in a subject QAOU. The edge offsetprocessing and band offset processing will be discussed laterspecifically.

(sao_offset)

Syntax sao_offset[sao_curr_depth][ys][xs][i] is syntax which representsa specific value of an offset to be added to each pixel included in asubject QAOU in offset filtering processing using an adaptive offsetfilter of this embodiment. In this embodiment, sao_offset may also besimply referred to as an “offset”.

Syntax sao_offset[sao_curr_depth][ys][xs][i] is specified by the index ias well as the arguments sao_curr_depth, ys, and xs. The index i is anindex for specifying a class, and is also indicated by class_idx. Theindex i takes one of integers 0 to 4 when the value ofsao_type_idx[sao_curr_depth][ys][xs] is one of 1 to 4 (that is, in thecase of an edge offset). The index i takes one of integers 0 to 16 whenthe value of sao_type_idx[sao_curr_depth][ys][xs] is 5 or 6 (that is, inthe case of a band offset). In whichever case, when i is 0, it indicatesthat an offset is not added, which will be discussed later.

The adaptive offset filter of this embodiment classifies a subject pixelincluded in a subject QAOU as one of the above-described plurality ofclasses, and adds an offset Offset associated with the class of thesubject pixel to the subject pixel. This will be discussed later.

Descriptor ue(v) shown in FIG. 3 indicates that syntax associated withthis descriptor is an unsigned numeric value and that variable-lengthcoding will be performed on the numeric value. Descriptor se(v) shown inFIG. 3 indicates that syntax associated with this descriptor is a signednumeric value and that variable-length coding will be performed on asign and an absolute value forming the numeric value.

(Video Decoding Device 1)

The video decoding device 1 of this embodiment will be described belowwith reference to FIGS. 1 and 5 through 13. The video decoding device 1partially includes a method defined in H. 264/MPEG-4. AVC, a method usedin KTA software, which is a joint development codec in VCEG (VideoCoding Expert Group), a method used in TMuC (Test Model underConsideration) software, which is a successor codec to the codec used inKTA software, and a technology used in HM (HEVC TestModel) software.

FIG. 5 is a block diagram illustrating the configuration of the videodecoding device 1. The video decoding device 1 includes, as shown inFIG. 5, a variable-length code decoder 13, a motion-vectorreconstructing unit 14, a buffer memory 15, an inter-prediction imagegenerator 16, an intra-prediction image generator 17, aprediction-method determining unit 18, aninverse-quantize-and-inverse-transform unit 19, an adder 20, adeblocking filter 41, an adaptive filter 50, and an adaptive offsetfilter 60. The video decoding device 1 is a device for generating avideo image #4 by decoding coded data #1.

The variable-length code decoder 13 decodes a prediction parameter PPconcerning each partition from the coded data #1. That is, concerning aninter-prediction partition, the variable-length code decoder 13 decodesa reference image index RI, an estimation motion-vector index PMVI, anda motion-vector residual MVD from the coded data #1, and supplies themto the motion-vector reconstructing unit 14. On the other hand,concerning an intra-prediction partition, the variable-length codedecoder 13 decodes (1) size specifying information which specifies thesize of the partition and (2) prediction-index specifying informationwhich specifies a prediction index from the coded data #1, and suppliesthem to the intra-prediction image generator 17. The variable-lengthcode decoder 13 also decodes CU information from the coded data andsupplies it to the prediction-method determining unit 18 (the flow of CUinformation is not shown). The variable-length code decoder 13 alsodecodes a quantized prediction residual QD concerning each block and aquantization parameter difference Δqp concerning a tree block includingthis block from the coded data #1 and supplies them to theinverse-quantize-and-inverse-transform unit 19. The variable-length codedecoder 13 also extracts QAOU information from the coded data #1 andsupplies the extracted QAOU information to the adaptive offset filter60.

The motion-vector reconstructing unit 14 reconstructs a motion vector myconcerning each inter-prediction partition from the motion-vectorresidual MVD concerning this partition and from a reconstructed motionvector mv′ concerning another partition. More specifically, themotion-vector reconstructing unit 14 (1) determines an estimation motionvector pmv from the reconstructed motion vector mv′ in accordance withan estimation method specified by the estimation motion vector indexPMVI, and (2) obtains the motion vector my by adding the motion-vectorresidual MVD to the estimation motion vector pmv. The reconstructedmotion vector mv′ concerning another partition may be read from thebuffer memory 15. The motion-vector reconstructing unit 14 supplies thereconstructed motion vector my to the inter-prediction image generator16, together with the associated reference image index RI. In the caseof an inter-prediction partition on which bidirectional prediction(weighted prediction) will be performed, the motion-vectorreconstructing unit 14 supplies two reconstructed motion vectors mv1 andmv2 to the inter-prediction image generator 16, together with associatedreference image indexes RI1 and RI2.

The inter-prediction image generator 16 generates a motion-compensatedimage me concerning each inter-prediction partition. More specifically,the inter-prediction image generator 16 generates a motion-compensatedimage me from a filtered decoded image P_FL′ specified by the referenceimage index RI supplied from the motion-vector reconstructing unit 14 byusing the motion vector my supplied from the motion-vectorreconstructing unit 14. The filtered decoded image P_FL′ is an imageobtained by performing deblocking processing by using the deblockingfilter 41, offset filtering processing by using the adaptive offsetfilter 60, and adaptive filtering processing by using the adaptivefilter 50 on a decoded image P. The inter-prediction image generator 16may read pixel values of individual pixels forming the filtered decodedimage P_FL′ from the buffer memory 15. The motion-compensated image megenerated by the inter-prediction image generator 16 is supplied to theprediction-method determining unit 18 as an inter-prediction imagePred_Inter. In the case of an inter-prediction partition on whichbidirectional prediction (weighted prediction) will be performed, theinter-prediction image generator 16 performs: (1) generating amotion-compensated image mc1 from a filtered decoded image P_FL1′specified by the reference image index RI1 by using a motion vector mv1;(2) generating a motion-compensated image mc2 from a filtered decodedimage P_FL2′ specified by the reference image index RI2 by using amotion vector mv2; and (3) generating an inter-prediction imagePred_Inter by adding an offset value to a weighted average of themotion-compensated image mc1 and the motion-compensated image mc2.

The intra-prediction image generator 17 generates a prediction imagePred_Intra concerning each intra-prediction partition. Morespecifically, the intra-prediction image generator 17 refers to aprediction mode decoded from the coded data #1 and assigns theprediction mode to a subject partition, for example, in a rasterscanning order. Then, the intra-prediction image generator 17 generatesa prediction image Pred_Intra from the decoded image P in accordancewith the prediction method represented by the prediction mode. Theintra-prediction image Pred_Intra generated by the intra-predictionimage generator 17 is supplied to the prediction-method determining unit18.

The intra-prediction image generator 17 also supplies the size of thesubject partition and intra-coding mode information IEM, which isinformation indicating the prediction mode assigned to the subjectpartition, to the adaptive filter 50.

The prediction-method determining unit 18 determines, on the basis ofthe CU information, whether each partition is an inter-predictionpartition to be subjected to inter prediction or an intra-predictionpartition to be subjected to intra prediction. If the subject partitionis an inter-prediction partition, the prediction-method determining unit18 supplies the inter-prediction image Pred_Inter generated by theinter-prediction image generator 16 to the adder 20 as the predictionimage Pred. If the subject partition is an intra-prediction partition,the prediction-method determining unit 18 supplies the intra-predictionimage Pred_Intra generated by the intra-prediction image generator 17 tothe adder 20 as the prediction image Pred.

The inverse-quantize-and-inverse-transform unit 19 performs: (1)inverse-quantizing the quantized prediction residual QD; (2) inverse-DCT(Discrete Cosine Transform) transform on a DCT coefficient obtained byperforming inverse quantization; and (3) supplying a prediction residualD obtained by performing inverse DCT transform to the adder 20. Whenperforming inverse quantization on the quantized prediction residual QD,the inverse-quantize-and-inverse-transform unit 19 determines aquantization step QP from the quantization parameter difference Δqpsupplied from the variable-length code decoder 13. The quantizationparameter qp can be determined by adding the quantization parameterdifference Δqp to a quantization parameter qp′ concerning a tree blockwhich has been subjected to inverse-quantization and inverse-DCTtransform immediately before the subject tree block, and thequantization step QP can be determined by calculating QP=2^(pq/6) fromthe quantization step qp. The generation of a prediction residual D bythe inverse-quantize-and-inverse-transform unit 19 is performed by usinga block (transform unit) as a unit.

The adder 20 generates a decoded image P by adding the predictionresidual D supplied from the inverse-quantize-and-inverse-transform unit19 to the prediction image Pred supplied from the prediction-methoddetermining unit 18.

When the difference between the pixel values of pixels adjacent to eachother with a block boundary or a CU boundary therebetween in the decodedimage P is smaller than a predetermined threshold, the deblocking filter41 performs deblocking processing on the block boundary or the CUboundary in the decoded image P, thereby smoothing an image in thevicinity of the block boundary or the CU boundary. The image subjectedto deblocking processing by the deblocking filter 41 is output to theadaptive offset filter 60 as a deblocked decoded image P_DB.

The adaptive offset filter 60 performs offset filtering processing usingan offset decoded from the coded data #1 on the deblocked decoded imageP_DB supplied from the deblocking filter 41, by using a QAOU as a unitof processing, thereby generating an offset-filtered decoded image P_OF.The generated offset-filtered decoded image P_OF is supplied to theadaptive filter 50. The specific configuration of the adaptive filter 60will be discussed later, and thus, an explanation thereof is omittedhere.

The adaptive filter 50 performs filtering processing using a filterparameter FP decoded from the coded data #1 on the offset-filtereddecoded image P_OF supplied from the adaptive offset filter 60, therebygenerating a filtered decoded image P_FL. The image subjected tofiltering processing by the adaptive filter 50 is output to the exterioras the filtered decoded image P_FL and is also stored in the buffermemory 15 in association with POC specifying information decoded fromthe coded data by the variable-length code decoder 13.

(Adaptive Offset Filter 60)

FIG. 1 is a block diagram illustrating the configuration of the adaptiveoffset filter 60. The adaptive offset filter 60 includes, as shown inFIG. 1, an adaptive offset filter information decoder 61 and an adaptiveoffset filter processor 62.

The adaptive offset filter information decoder 61 includes, as shown inFIG. 1, an offset information decoding section 611, a QAOU structuredecoding section 612, and an offset attribute setting section 613.

The offset information decoding section 611 refers to QAOU informationincluded in the coded data #1 and decodes offset information OI includedin the QAOU information. The values sao_type_idx[sao_curr_depth][ys][xs]and sao_offset[sao_curr_depth][ys][xs][i] obtained by decoding theoffset information OI are supplied to an offset information storagesection 621 in association with the arguments (sao_curr_depth, ys, xs)and the arguments (sao_curr_depth, ys, xs, i), respectively.

The QAOU structure decoding section 612 determines a QAOU splitstructure by decoding sao_split_flag[sao_curr_depth][ys][xs] included inthe QAOU information, and supplies QAOU structure information indicatingthe determined QAOU split structure to the offset information storagesection 621.

The offset attribute setting section 613 determines the offset bit depth(also referred to as “SAO_DEPTH) and the offset value range. In thiscase, the offset bit depth is determined from the pixel bit depth (alsoreferred to as “PIC_DEPTH”) (not shown) input into the offset attributesetting section 613. The pixel bit depth represents a range of pixelvalues forming an image input into the adaptive offset filter 60 as thebit width. When the pixel bit depth is N bits, the pixel values take arange of 0 to 2^(N)−1. For example, when the pixel bit depth is 8 bits,the pixel values may take range of 0 to 255. If the image input into theadaptive offset filter 60 is a decoded image obtained by the videodecoding device 1 or a locally decoded image obtained by the videocoding device 2, the bit depth of the decoded image or the locallydecoded image is used. The precision of an image input into the adaptiveoffset filter 60 and the precision of an image output from the adaptiveoffset filter 60 are represented by the pixel bit depth.

As the pixel bit depth, a value determined by using syntax in the codeddata #1 can be decoded and used. For example, in the case of H. 264/AVC,the pixel bit depth can be determined by using bit_depth_luma_minus8 ina sequence parameter set. In another mode, the pixel bit depth may bedetermined for each unit area to be processed, in which case, the pixelbit depth is decoded from the header of each unit area. The pixel bitdepth in each unit area may be included in the parameter information orthe header in the coded data #1, which is used for decoding an imageinput into the adaptive offset filter 60, or may be included as part ofthe QAOU information. If the pixel bit depth is included in theparameter information or the header, which is used for decoding an inputimage, it may be included in a picture parameter header, a slice header,an LCU, or a CU. If the pixel bit depth is included in part of the QAOUinformation, it may be included in a leaf QAOU or in QAOU information ofa predetermined level (for example, the highest level QAOU or the firstlevel QAOU). Concerning the coding of the pixel bit depth, it ispreferable that it is coded as a difference value from eight bits. Ifpart of the QAOU information is used, instead of using the pixel bitdepth, it is also suitable that a shift value itself is coded. In thiscase, the shift value may be suitably coded only when it exceeds eightbits. The shift value is a bit shift amount which is necessary forperforming inverse quantization. As a parameter used for inversequantization, a step value may be used instead of the shift value. Inthis case, inverse quantization of an offset is performed by multiplyingthe offset by a step value, and quantization of an offset is performedby dividing the offset by a step value.

The offset bit depth is a value indicating the precision of an offsetcoded in the coded data #1. The offset included in the coded data #1 isa quantized value, and this quantized offset is decoded, and is thensubjected to inverse quantization and is transformed into the bit depthcorresponding to the pixel bit depth by an offset determining unit 625,which will be discussed later. Then, the inverse-quantized offset isadded by an offset adder 626, which will be discussed later. The offsetbit depth has a value equal to or smaller than the pixel bit depth. Ifthe offset bit depth is smaller than the pixel bit depth by k bits, itmeans that the value of a coded offset is a value quantized by an offsetquantization step of 2^(k). Transformation from the decoded bit depth(offset bit depth) into the pixel bit depth is performed by the offsetdetermining unit 625, which will be discussed later.

The offset attribute setting section 613 sets an offset value range byusing the determined offset bit depth. The offset attribute settingsection 613 further sets a shift value from the offset pixel bit depthand the offset value range. The set offset value range and the set shiftvalue are supplied to the offset information storage section 621 and theoffset information decoding section 611. The determination of the offsetbit depth and the setting of the shift value are performed by using oneof patterns S1 through S6, which will be discussed later. The setting ofthe offset value range is performed by using one of patterns C1 and C2,which will be discussed later. The offset value range and the shiftvalue, which are both used as attributes of an offset, are referred toas “offset attributes”. The adaptive offset filter processor 62includes, as shown in FIG. 1, the offset information storage section621, a QAOU controller 622, an offset-type determining section 623, aclassifying section 624, the offset determining section 625, and anoffset adder 626.

The offset information storage section 621 manages and stores the offsettype specified for each QAOU and specific values of offsets with respectto individual classes that can be selected for the specified offsettype, on the basis of the QAOU structure information,sao_type_idx[sao_curr_depth][ys][xs], andsao_offset[sao_curr_depth][ys][xs][i]. The offset information storagesection 621 has a map memory and a list memory.

In the map memory, a QAOU index assigned to each offset minimum unit(also referred to as “QAOMU: Quad Adaptive Offset Minimum Unit”), whichis determined by the split depth, is stored. The QAOU index will bediscussed later. The map memory will now be described below withreference to part (a) of FIG. 6. Part (a) of FIG. 6 illustrates examplesof QAOU indexes stored in the map memory, and shows QAOMUs having asplit depth of three and forming the unit of processing (for example, anLCU) and QAOU indexes assigned to QAOMUs. In part (a) of FIG. 6, indexes0 to 9 specify QAOUs in a simple manner without considering the QAOUsplit depth. In the example shown in part (a) of FIG. 6, a QAOUspecified by QAOU index=1 is indicated by QAOUI. In part (a) of FIG. 6,the thin lines indicate QAOMU boundaries, while the thick lines indicateQAOU boundaries.

As shown in part (a) of FIG. 6, QAOU0 is constituted by four QAOMUs, and0 is assigned to these four QAOMUs as the QAOU index. QAOU3 isconstituted by one QAOMU, and 3 is assigned to this QAOMU as the QAOUindex. In this manner, in the map memory, QAOU indexes assigned to theindividual QAOMUs are stored.

In the list memory, each QAOU index, the offset type associated witheach QAOU index, and specific values of offsets with respect to classesthat can be selected for this offset type are stored in association witheach other. In the offset information storage section 621, offsetshaving a restricted range of values set by the offset attribute settingsection 613 are stored.

Part (b) of FIG. 6 illustrates an example of information stored in thelist memory, and shows an offset type associated with each of the QAOUindexes 0 to 9 and offsets with respect to individual classes that canbe selected for each offset type. In part (b) of FIG. 6, “xxx” indicatesspecific numeric values that may be different from each other.

In part (b) of FIG. 6, “BO_1” indicates an offset type specified bysao_type_idx=5. “EO_1” indicates an offset type specified bysao_type_idx=1. In this manner, edge offsets, which are of an offsettype specified by sao_type_idx=1, 2, 3, 4, are also indicated by EO_1,2, 3, 4, respectively, and band offsets, which are of an offset typespecified by sao_type_idx=5, 6, are also indicated by BO_1, 2,respectively.

As shown in part (b) of FIG. 6, when the offset type is a band offset, atotal of sixteen offsets, that is, the offset 1 through the offset 16,are stored in the list memory for this offset type. The offset 1 throughthe offset 16 are values specified bysao_offset[sao_curr_depth][ys][xs][1] throughsao_offset[sao_curr_depth][ys][xs][16], respectively, when the value ofsao_type_idx[sao_curr_depth][ys][xs] is 5 or 6. On the other hand, whenthe offset type is an edge offset, a total of four offsets, that is, theoffset 1 through the offset 4, are stored in the list memory for thisoffset type. The offset 1 through the offset 4 are values specified bysao_offset[sao_curr_depth][ys][xs][1] throughsao_offset[sao_curr_depth][ys][xs][4], respectively, when the value ofsao_type_idx[sao_curr_depth][ys][xs] is one of 1, 2, 3, and 4. No valueis stored in the offset 5 through the offset 16.

The memory size of each offset stored in the list memory is determinedby the offset value range supplied from the offset attribute settingsection 613. When the offset value range is −2⁴ to 2⁴−1, each offset canbe represented by five bits, and thus, a memory size having five bits isrequired.

A QAOMU number is appended to each QAOMU, and the QAOMUs can bedistinguished from each other by the QAOMU numbers. Hereinafter, QAOMUhaving a QAOMU number NQ will also be referred to as “QAOMUN_(Q)”.

QAOU indexes are QAOU block numbers which are serially specified whenthe split depth is 0 to the maximum. If the maximum split depth is four,values of 0 to 340 are specified for all the blocks (1+4+16+64+256=341)of all the split depths by the QAOU indexes, as shown in FIG. 7.

Since it is not necessary to store all the QAOU offsets of all the splitdepths in the offset information storage section 621 used in the videodecoding device 1, there is no need to secure 341 memory areas, andinstead, it is sufficient if the same number of memory areas as that ofQAOUs which are actually used in the structure specified by the inputQAOU information are provided. When the maximum split depth is four, thenumber of blocks is 256 or smaller, and thus, the provision of 256 mapmemories and 256 list memories is sufficient. In this case, as the QAOUindexes used in the offset information storage section 621, uniqueindexes for identifying leaf QAOUs, for example, indexes from 0 to 255or smaller which are incremented by one every time a leaf QAOU isdecoded, are used. It is also possible to store, in the offsetinformation storage section 621, a map list by using QAOU informationcorresponding to the maximum split depth as a unit. In this case, 0 to255 are used as the QAOU index numbers used in the offset informationstorage section 621, and they correspond to QAOMU numbers 85 to 340 inFIG. 7.

Parts (a) through (e) of FIG. 7 show examples of QAOMU numbers appendedto QAOMUs included in the unit of processing (for example, an LCU). Part(a) of FIG. 7 shows a QAOMU number assigned to a QAOMU having a splitdepth of 0. Part (b) of FIG. 7 shows QAOMU numbers assigned to QAOMUshaving a split depth of one. Part (c) of FIG. 7 shows QAOMU numbersassigned to QAOMUs having a split depth of two. Part (d) of FIG. 7 showsQAOMU numbers assigned to QAOMUs having a split depth of three. Part (e)of FIG. 7 shows QAOMU numbers assigned to QAOMUs having a split depth offour.

The QAOU controller 622 controls the individual components included inthe adaptive offset filter processor 62. The QAOU controller 622 alsorefers to the QAOU structure information, and then splits the deblockeddecoded image P_DB into one or a plurality of QAOUs and scans theindividual QAOUs in a predetermined order. The QAOU controller 622 alsosupplies a QAOMU number representing a subject QAOMU to the offset-typedetermining section 623.

The offset-type determining section 623 refers to the map memory and thelist memory of the offset information storage section 621 and determinesthe offset type specified by the QAOMU number supplied from the QAOUcontroller 622. The offset-type determining section 623 also suppliesthe determined offset type to the classifying section 624.

The classifying section 624 classifies each pixel included in thesubject QAOU as one of the classes that can be selected for the offsettype supplied from the offset-type determining section 623. Theclassifying section 624 also supplies the offset type and the classindex indicating the class of each pixel to the offset determiningsection 625. Specific classifying processing performed by theclassifying section 624 will be discussed later, and thus, anexplanation thereof is omitted here.

The offset determining section 625 refers to the list memory of theoffset information storage section 621, and determines, for each pixelincluded in the subject QAOU, an offset specified by the offset type andthe class index supplied from the classifying section 624. The offsetdetermining section 625 includes an offset reverse shifter (not shown)which performs bitwise left shift on an offset by an amount equal to ashift value set by the offset attribute setting section 613. The offsetreverse shifter performs inverse quantization on the offset so that theoffset bit depth may match the pixel bit depth. By performing suchinverse quantization, in addition processing performed by the offsetadder 626, which will be discussed later, the offset can be added to thepixel value by using the same bit depth. The inverse-quantized offsetfor each pixel is supplied to the offset adder 626.

The offset adder 626 adds the offset supplied from the offsetdetermining section 625 to the associated pixel which forms the subjectQAOU of the deblocked decoded image P_DB. The offset adder 626 outputsan image obtained by performing processing on all the QAOUs included inthe deblocked decoded image P_DB as an offset-filtered decoded imageP_OF.

FIG. 8 is a flowchart of a flow of processing performed by the adaptiveoffset filter processor 62.

(Step S101)

First, the QAOU controller 622 obtains QAOU structure information fromthe offset information storage section 621.

(Step S102)

Then, the QAOU controller 622 starts a first loop using the QAOMU numberof a subject QAOMU as a loop variable.

(Step S103)

The QAOU controller 622 supplies the QAOMU number to the offset-typedetermining section 623. Under the control of the QAOU controller 622,the offset-type determining section 623 reads an offset type specifiedby the QAOMU number supplied from the QAOU controller 622 from the mapmemory and the list memory of the offset information storage section621. The offset-type determining section 623 also supplies the readoffset type to the classifying section 624.

(Step S104)

Then, the QAOU controller 622 starts a second loop using the pixelnumber of each pixel included in the subject QAOMU as a loop variable.The pixel numbers are used for distinguishing pixels included in thesubject QAOMU from each other, and numbers assigned to pixels includedin the subject QAOMU in a predetermined scanning order, for example, maybe used. Instead of using the pixel numbers, x and y coordinates of eachpixel included in the subject QAOMU may be used as loop variables.

(Step S105)

Then, under the control of the QAOU controller 622, the classifyingsection 624 classifies the subject pixel as one of a plurality ofclasses that can be selected for the offset type supplied from theoffset-type determining section 623. The classifying section 624 alsosupplies the offset type and the class index indicating the class of thesubject pixel to the offset determining section 625.

(Step S106)

Then, under the control of the QAOU controller 622, the offsetdetermining section 625 reads an offset to be added to the subject pixelfrom the offset information storage section 621. That is, the offsetdetermining section 625 reads the offset specified by the offset typeand the class index supplied from the classifying section 624. Theoffset determining section 625 also inverse-quantizes the offset byexecuting bitwise left shift on the offset determined for the subjectpixel by an amount equal to a shift value supplied from the offsetattribute setting section 613. The offset determining section 625 thensupplies the inverse-quantized offset to the offset adder 626.

(Step S107)

Then, under the control of the QAOU controller 622, the offset adder 626adds the offset supplied from the offset determining section 625 to thepixel value of the subject pixel which forms the deblocked decoded imageP_DB.

(Step S108)

This step is the termination of the second loop.

(Step S109)

This step is the termination of the first loop.

If it is found in step S103 that the offset read by the offset-typedetermining section 623 is 0 (offset type=0), the QAOU controller 622controls the offset adder 626 so that the offset adder 626 will not addan offset to any of the pixels forming the subject QAOMU.

If it is found in step S105 that the subject pixel is classified asclass 0 (class index=0), the QAOU controller 622 controls the offsetadder 626 so that the offset adder 626 will not add an offset to thesubject pixel.

(The Number of Bits Necessary for Storing Offset)

The number of bits necessary for storing offsets (sao_offset) will nowbe discussed below. When the pixel bit depth is ten bits and the offsetbit depth is nine bits, the offset may take a range of values of −2⁹ to2⁹−1, and the number of bits per offset is as large as ten bits. If anoffset having such a large number of bits is decoded, it is necessarythat the offset information storage section 621 have the followingmaximum memory size for storing offsets for each picture:(the total number of QAOMUs per picture)×(the number of classes)×(thenumber of bits per offset)=256×16×10 (bits)=40960 (bits).

Although the total number of QAOMUs per picture is 256 in the decodingdevice, 341 QAOMUs are used in a coding device, which will be discussedlater, and thus, an even larger memory is required. In this manner, ifthe value range that can be taken with the offset bit depth is usedwithout restricting the offset value range, a large memory size isrequired for storing offsets since each offset has a large number ofbits.

(Relationship Between SAO_DEPTH and Coding Efficiency)

SAO_DEPTH and PIC_DEPTH are closely related to each other in terms ofquantization errors. The bit depth of an image output from the adaptiveoffset filter 60 is represented by the pixel bit depth PIC_DEPTH, andSAO_DEPTH represents the bit depth of an offset to be added to a pixel.Accordingly, even if an offset having a higher precision than that ofthe pixel bit depth is used, it is discarded in the output process. Itis thus preferable that SAO_DEPTH, which indicates the offset precision,is set to be equal to or lower than PIC_DEPTH. On the other hand, ifSAO_DEPTH is lower than PIC_DEPTH, an input image is corrected merelywith a precision lower than the precision (PIC_DEPTH) with which theinput image could be corrected by using a filter, thereby decreasing thefiltering effect.

In contrast, if the offset precision SAO_DEPTH is higher, the amount ofdata required to code offsets increases. Generally, as is seen from thefact that the coding efficiency is optimized by minimizing therate-distortion cost D+λR represented by using the amount of data (rate)R required for coding, the distortion D of an input image, and theweight λ, the offset precision decreases the distortion D and increasesthe rate R. Thus, concerning the level of precision, there is a tradeoffbetween a decrease in the distortion D and an increase in the rate R inwhich a specific optimal value of the precision is provided.

Further, in this embodiment, by restricting a quantized offset to anoffset value range that can be represented by a certain bit width, it ispossible to reduce the bit width for storing a quantized offset in theoffset information storage section 621. With this arrangement, theeffect of reducing the memory size can be obtained, compared with theabove-described restriction is not imposed. However, if the offset valuerange is excessively restricted, the effect of correcting distortion ofa decoded image by using an offset is decreased, and it is not possibleto remove distortion of a decoded image in offset addition processing,which may decrease the coding efficiency. Thus, it is preferable thatthe offset value range is set in an optimal range in such a degree asnot to decrease the coding efficiency.

In this embodiment, the offset attribute setting section 613 sets theoffset bit depth and the offset shift value by using one of thefollowing patterns S1 through S6 and sets the offset value range byusing one of the following patterns C1 through C3.

(Pattern S1)

In pattern S1, as shown in part (a) of FIG. 9, the offset bit depthSAO_DEPTH is set to be equal to the pixel bit depth PIC_DEPTH. Since themaximum value of the offset precision is equal to the pixel bit depth,an offset is coded with the maximum precision in the pattern S₁.

(Pattern S2)

In pattern S2, as shown in part (b) of FIG. 9, when PIC_DEPTH is tenbits or smaller, SAO_DEPTH is set to be equal to PIC_DEPTH, and whenPIC_DEPTH is eleven bits or greater, SAO_DEPTH is set to be ten bits. Inpattern S2, the upper limit of the offset bit depth is set to be tenbits. The inventors have found that, when the quantization step QP of adecoded image is small (when the bit rate is high), the codingefficiency becomes higher if the offset bit depth is equal to the pixelbit depth than a case in which the offset bit depth is smaller than thepixel bit depth, and that, conversely, when the quantization step QP islarge, the coding efficiency becomes higher if the offset bit depth issmaller than the pixel bit depth than a case in which the offset bitdepth is equal to the pixel bit depth. It has been found through theinventors' experiments that, when the quantization parameter qp rangedfrom 12 to 27, the coding efficiency was improved if the offset bitdepths were determined as in pattern S2, compared with a case in whichthe offset bit depth was eight bits when the pixel bit depth was ninebits or smaller and the offset bit depth was nine bits when the pixelbit depth was ten bits or greater. Accordingly, in pattern S2, bychanging the dependency of the offset bit depth on the pixel bit depthin the boundary of ten bits, the amount of data required to code offsetscan be decreased, compared with a case in which the offset bit depth isequal to the pixel bit depth, as in pattern S₁, thereby achieving a highcoding efficiency.

(Pattern S3)

In pattern S3, as shown in part (c) of FIG. 9, when PIC_DEPTH is ninebits or smaller, SAO_DEPTH is set to be equal to PIC_DEPTH, and whenPIC_DEPTH is ten bits or greater, SAO_DEPTH is set to be nine bits. Inpattern S3, the upper limit of the offset bit depth is set to be ninebits. In pattern S3 as well as in pattern S2, the amount of datarequired to code offsets can be decreased, thereby achieving a highcoding efficiency.

(Pattern S4)

In pattern S4, as shown part (d) of FIG. 9, when PIC_DEPTH is ten bitsor smaller, SAO_DEPTH is set to be equal to PIC_DEPTH, and whenPIC_DEPTH is eleven bits or greater, SAO_DEPTH is set to be10-floor((PIC_DEPTH−10)/STEP). The function floor(x) is a function thatprovides the largest integer not greater than x. In pattern S4, when thepixel bit depth is eleven bits or greater, every time the pixel bitincrease (decreases) by STEP bits, the offset bit depth increases(decreases) by one bit. Part (e) of FIG. 9 shows a case in which STEP istwo in pattern S4, and every time the pixel bit depth increases by twobits, the offset bit depth increases by one bit. With thisconfiguration, it is possible to more flexibly respond to the level ofthe bit rate than in pattern S2 or S3 while also considering the levelof the pixel bit depth.

For all of pattern S1 through pattern S4, the shift value is representedby the difference value PIC_DEPTH−SAO_DEPTH between PIC_DEPTH andSAO_DEPTH. By changing the offset bit depth and the shift value as inthe above-described pattern S1 through S4, the offset bit depth and theshift value can be set without increasing the memory size or the amountof processing, thereby making it possible to improve the codingefficiency.

(Pattern S5)

In pattern S5, the offset bit depth is explicitly coded. Morespecifically, the difference between the offset bit depth and apredetermined value is coded. It is suitable that the predeterminedvalue is eight or the pixel bit depth. If the predetermined value iseight, SAO_DEPTH−8 is coded. If the predetermined value is the pixel bitdepth, PIC_DEPTH−SAO_DEPTH is coded. The offset bit depth may be safelycoded as various items of parameter information or the header of codeddata or as part of QAOU information. If the bit depth is coded as partof QAOU information, it may be safely included in a leaf QAOU or in QAOUinformation of a predetermined level (for example, the highest levelQAOU or the first level QAOU). By coding the bit depth as part of codeddata, the bit depth can be set to be an optimal value in the decodingdevice and in the coding device, thereby obtaining the effect ofmaximizing the coding efficiency. If the bit depth is coded as QAOUinformation, the bit depth is changed in accordance with the QAOU depthsao_curr_depth, thereby making it possible to reduce the memory size forstoring offsets. It is likely that many offsets will appear whensao_curr_depth is large. Accordingly, by setting the bit depth to besmall when sao_curr_depth is large and by setting the bit depth to belarge when sao_curr_depth is small, it is possible to reduce therequired memory size. For example, it is suitable that, whensao_curr_depth=0 to 1, the offset bit depth is set to be equal to thepixel bit depth (pattern S1) and when sao_curr_depth=2 to 4, the upperlimit is set for the offset bit depth (pattern S2). The bit width iscoded in this manner. It is also suitable that a flag indicating whetherthe offset bit width will be coded in response to the QAOU depth or onlyone offset bit depth will be used regardless of the QAOU depth may becoded, thereby switching between a case in which the offset bit depth iscoded in response to the QAOU depth and a case in which only one offsetbit depth is used regardless of the QAOU depth.

(Pattern S6)

Without coding the bit depth explicitly, the bit depth is determined inaccordance with sao_curr_depth. For example, it is suitable that, whensao_curr_depth=0 to 1, the offset bit depth is set to be equal to thepixel bit depth (pattern S1), and when sao_curr_depth=2 to 4, the upperlimit is set for the offset bit depth (pattern S2).

(Pattern C1)

In pattern C1, the offset value range is set in accordance withSAO_DEPTH. Hereinafter, the maximum bit length representing the offsetvalue range is indicated by CLIP_BIT. More specifically, by calculatingCLIP_BIT=SAO_DEPTH−K, the offset value range is determined to be−2^(CLIP_BIT−1) to 2^(CLIP_BIT−1)−1. It has been found through theinventors' experiments that K=4 is suitable. That is, it has beenvalidated that, in the case of K=4, the coding efficiency was notdecreased even if the offset range was restricted by the offset valuerange. K=4 is suitable when the pixel bit depth is eight, which is mostcommonly used. When the pixel bit depth is eight, the offset bit depthSAO_DEPTH is also eight, and thus, CLIP_BIT=8−K=4. If one offset can bestored by using four bits, in software, for example, which handles onebyte constituted by eight bits as the unit, two offsets can be packedand stored in one byte, thereby easily reducing the memory size.

(Pattern C2)

In pattern C2, the offset value range is set independently of SAO_DEPTH.More specifically, CLIP_BIT is set to be eight, and the offset valuerange is set to be −2⁷ to 2⁷−1.

Alternatively, generally, a constant N which is independent of SAO_DEPTHmay be used, and CLIP_BIT may be set to be N. When the offset valuerange is set independently of SAO_DEPTH, it is preferable that theoffset value range is set to be smaller than the offset bit depth inorder to achieve the effect of reducing the memory size.

(Pattern C3)

In pattern C3, the offset value range is determined in accordance withthe QAOU hierarchical depth. It is suitable that, when sao_curr_depth issmall (for example, 0 to 1), the offset value range is determinedindependently of the offset bit depth, and when sao_curr_depth is large(for example, 2 to 4), the offset value range is determined inaccordance with the offset bit depth. If the offset value range isdetermined independently of the offset bit depth, CLIP_BIT may be set tobe eight (pattern C2). If the offset value range is determined inaccordance with the offset bit depth, CLIP_BIT may be set to beSAO_DEPTH−K bits (pattern C1). If the offset bit length is determined inaccordance with the QAOU hierarchical depth, a fixed bit number may besafely used for CLIP_BIT. For example, CLIP_BIT=4 is suitable.

(First Specific Example of the Number of Offset Bits)

A first specific example of the number of bits of an offset (sao_offset)according to this embodiment will be discussed below. In this example, acase in which the pixel bit depth is set to be ten bits, and the shiftvalue is set by using pattern S2 and the offset value range is set byusing pattern C1 will be discussed. In pattern S2, the offset bit depthis set to be ten bits, and, in pattern C1, the offset value range is setto be 10−4=6 bits. If the number of bits per offset is six bits, itmeans that the offset values are restricted to −32 to 31. In thisexample, as the memory size for storing offsets in the offsetinformation storage section 621, the following maximum size for eachpicture is sufficient: (the total number of QAOMUs per picture)×(thenumber of classes)×(the number of bits per offset)=256×16×6 (bits)=24576(bits).

Thus, with the configuration in which the offset value range is set asin this example, the memory size necessary for the offset informationstorage section 621 can be reduced to substantially ⅗ of the memory sizeof an example of the related art.

Since the amount of data required to code offsets included in the codeddata #1 can be reduced, it is possible to improve the coding efficiency.Additionally, since excessive offsets are not added, a suitable level ofimage quality can be guaranteed.

(Second Specific Example of the Number of Offset Bits)

A second specific example of the number of bits of an offset(sao_offset) according to this embodiment will be discussed below. Inthis example, a case in which the pixel bit depth is set to be ten bits,and the shift value is set by using pattern S2 and the offset valuerange is set by using pattern C2 will be discussed. In pattern S2, theoffset bit depth is set to be ten bits, and, in pattern C2, the offsetvalue range is set to be eight bits. If the number of bits per offset iseight bits, it means that the offset values are restricted to −128 to127. In this example, as the memory size for storing offsets in theoffset information storage section 621, the following maximum size foreach picture is sufficient:(the total number of QAOMUs per picture)×(the number of classes)×(thenumber of bits per offset)=256×16×8 (bits)=32768 (bits).

Thus, with the configuration in which the number of bits of an offset isset as in this example, the memory size necessary for the offsetinformation storage section 621 can be reduced to substantially ⅘ of thememory size of an example of the related art.

Since the amount of data required to code offsets included in the codeddata #1 can be reduced, it is possible to improve the coding efficiency.Additionally, since excessive offsets are not added, a suitable level ofimage quality can be guaranteed.

If the number of offset bits is restricted, the amount of data requiredto code offset information included in the coded data #1 can be reduced.However, if the number of offset bits is excessively restricted, theeffect of providing an adaptive offset filter is decreased, therebyincreasing the amount of data required to code residual data (the pixelvalue of a residual image) included in the coded data.

(Third Specific Example of the Number of Offset Bits)

A third specific example of the number of bits of an offset (sao_offset)will be discussed below. In any one of the patterns C1 and C2 of thisexample, the value to be set for the number of bits of an offset ischanged depending on whether the offset type is an edge offset (offsettype is one of 1 to 4) or a band offset (offset type is 5 or 6). In thisexample, a method for setting the shift value by using pattern S2 andfor setting the offset value range of an edge offset by using pattern C2and the offset value range of a band offset by using pattern C1 when thepixel bit depth is set to be ten bits will be discussed.

When the pixel bit depth is ten bits, in pattern S2, the offset bitdepth is set to be ten bits. The value range of an offset belonging tothe edge offset type (hereinafter referred to as an “edge-offsetoffset”) is set to be eight bits in pattern C2. The value range of anoffset belonging to the band offset type (hereinafter referred to as a“band-offset offset”) is set to be six bits in pattern C1. Moregenerally, when the number of bits of an edge-offset offset is N bitsand the number of bits of a band-offset offset is M bits, the numbers ofbits of offsets are determined so that the condition N≥M can besatisfied.

The memory size required to be secured in the offset information storagesection 621 is the number of bits represented by (the number of classesof an offset type x the number of offset bits) for each QAOU.Accordingly, the number of bits of a band offset, which has a greaternumber of classes than an edge offset, is set to be smaller, therebymaking it possible to effectively utilize the memory area for storingoffsets in the offset information storage section 621.

In this manner, by varying the number of bits of an offset in accordancewith the offset type, the coding efficiency can be improved withoutrequiring an excessive memory size of the offset information storagesection 621. The memory area can also be utilized most effectively.

When the threshold th for restricting values that can be taken as anoffset is greater than 2^(m-1) but not greater than 2^(m), m-bitfixed-length coding/decoding method may be used as the coding method forcoding the offset. The variable-length coding/decoding method, such asTruncated unary coding or Truncated Rice coding having th as the maximumvalue, may be used. The above-described maximum value th is determinedby the offset value range supplied from the offset attribute settingsection 613. The video decoding device 1 is capable of decoding offsetswhich have been coded as described above.

With the above-described configuration, in the offset attribute settingsection 613, the offset bit depth, the offset value range, and the shiftvalue are set. In the adaptive offset filter information decoder 61, aquantized offset having a value within the offset value range is decodedand is stored in the offset information storage section 621 in which astorage area having a bit width equal to or larger than the offset valuerange is secured for each offset. In this embodiment, it ischaracterized in that the offset value range is determined in accordancewith the offset bit depth. The offset bit depth is determined inaccordance with the pixel bit depth. Accordingly, in this embodiment, itis also characterized in that the offset bit depth is determined inaccordance with the pixel bit depth.

The adaptive offset filter information decoder 61 of the firstembodiment may include a storage section for storing decoded offsetstherein and an inverse-quantizing section which inverse-quantizesoffsets obtained from the storage section. Inverse-quantizationprocessing performed by the offset determining section 625 may beomitted. In this case, it is characterized in that the storage sectionstores an offset which is restricted to the offset value range set bythe offset attribute setting section 613 and in that theinverse-quantizing section performs inverse-quantization processing byperforming bitwise left shift on an offset in accordance with the shiftvalue set by the offset attribute setting section 613.

The offset information decoding section 611 decodes, from the coded data#1, each offset to be referred to by the offset adder 626, which adds anoffset to the pixel value of each pixel forming an input imageconstituted by a plurality of unit areas. In other words, the offsetinformation decoding section 611 includes offset decoding means (notshown) for decoding an offset having an offset value range and a shiftvalue which are set in accordance with the pixel bit depth and which isrestricted to the offset value range.

The adaptive offset filter 60 of this embodiment is an image filteringdevice which adds an offset to the pixel value of each pixel forming aninput image constituted by a plurality of unit areas. In other words,the adaptive offset filter 60 may be an image filtering device includingthe offset attribute setting section 613 that refers to offset-typespecifying information included in coded data and sets offset attributesin a subject unit area, the offset information decoding section 611 thatdecodes an offset having a bit width corresponding to an offset valuerange included in the offset attributes set as described above, and theoffset adder 626 that adds the offset to the pixel value of each pixelforming the input image.

In addition to the above-described offset decoding means, the offsetinformation decoding section 611 may include determining means fordetermining the offset type to which a subject unit area belongs fromamong a plurality of offset types, and offset decoding means fordecoding offsets having different bit widths depending on the offsettype determined by the determining means.

The above-described offset-type specifying information includesinformation concerning the bit depth of pixel values in each unit areaof the input image, and the offset information decoding section 611 maydecode an offset having a bit width corresponding to the bit depth ofthe pixel values.

A description will now be given of specific examples of classifyingprocessing performed by the classifying section 624. It is preferablethat the classifying section 624 performs, among the following examplesof classifying processing, classifying processing corresponding toclassifying processing performed by the video coding device that hasgenerated the coded data #1.

(First Example of Classifying Processing of Classifying Section 624)

A first example of classifying processing performed by the classifyingsection 624 will be described below with reference to parts (a) through(d) of FIG. 10 through FIG. 12.

(When Offset Type is One of 1 to 4 (Edge Offset))

When the offset type supplied from the offset-type determining section623 is one of 1 to 4, the classifying section 624 determines whether ornot there is an edge near a subject pixel and, if there is an edge,determines the type of edge. The classifying section 624 then classifiesthe subject pixel as one of a plurality of classes in accordance withthe determination results.

More specifically, the classifying section 624 first determines the signof the difference between the pixel value pic[x] of a subject pixel xand each of the pixel values pic[a] and pic[b] of two pixels a and bwhich are adjacent to the subject pixel or which have the same vertex asthe subject pixel. That is, the classifying section 624 calculates:Sign(pic[x]−pic[a]) andSign(pic[x]−pic[b]):where Sign(z) is a function which takes the following values:Sign(z)=+1 (when z>0)Sign(z)=0 (when z=0)Sign(z)=−1 (when z<0).Specifically, a determination as to which pixels are used as the pixel aand the pixel b is dependent on the offset type, and is made as follows.

-   -   When offset type is 1 (sao_type_idx=1)

As shown in part (a) of FIG. 10, the pixel positioned immediately on theleft side of the subject pixel x is set to be the pixel a, and the pixelpositioned immediately on the right side of the subject pixel x is setto be the pixel b.

-   -   When offset type is 2 (sao_type_idx=2)

As shown in part (b) of FIG. 10, the pixel positioned immediately on thetop side of the subject pixel x is set to be the pixel a, and the pixelpositioned immediately on the bottom side of the subject pixel x is setto be the pixel b.

-   -   When offset type is 3 (sao_type_idx=3)

As shown in part (c) of FIG. 10, the pixel having the same vertex asthat of the top left side of the subject pixel x is set to be the pixela, and the pixel having the same vertex as that of the bottom right sideof the subject pixel x is set to be the pixel b.

-   -   When offset type is 4 (sao_type_idx=4)

As shown in part (d) of FIG. 10, the pixel having the same vertex asthat of the bottom left side of the subject pixel x is set to be thepixel a, and the pixel having the same vertex as that of the top rightside of the subject pixel x is set to be the pixel b.

Part (a) of FIG. 11 shows graphs indicating the magnitude relationsbetween the pixel value pic[x] of the subject pixel x and the pixelvalue of the pixel a or b and also shows the values of the function Signin accordance with the magnitude relation. In the graphs shown in part(a) of FIG. 11, the black circle with pic[x] indicates the pixel valueof the subject pixel x, and the black circle without pic[x] indicatesthe pixel value of the subject pixel a or b. The vertical direction inthe graphs shown in part (a) of FIG. 11 indicates the magnitude of thepixel value.

Then, the classifying section 624 finds EdgeType according to thefollowing mathematical equation (1-1) on the basis ofSign(pic[x]−pic[a]) and Sign(pic[x]−pic[b]).EdgeType=Sign(pic[x]−pic[a])+Sign(pic[x]−pic[b])+2  (1-1)

Part (b) of FIG. 11 shows graphs indicating the magnitude relationsbetween the pixel value of the subject pixel x and each of the pixelvalues of the pixel a and the pixel b and also shows the values ofEdgeType in accordance with the magnitude relation. In part (b) of FIG.11, the center black circle in each graph indicates the pixel value ofthe subject pixel x, and the black circles on both sides indicate thepixel values of the pixel a and the pixel b. The vertical direction inthe graphs shown in part (b) of FIG. 11 indicates the magnitude of thepixel value.

Then, the classifying section 624 determines as follows, on the basis ofthe determined EdgeType, the class index (class_idx) of the class towhich the subject pixel x belongs.class_idx=EoTbl[EdgeType]where EoTbl[EdgeType] is a transform table used for determiningclass_idx from EdgeType. A specific example of the transform table EoTblis shown in part (d) of FIG. 11.

As shown in part (d) of FIG. 11, when there is no edge in an areaconstituted by the subject pixel x, the pixel a, and the pixel b(hereinafter, such a case will also be referred to as a case in which anarea is flat), the subject pixel x is classified as class 0(class_idx0). Part (c) of FIG. 11 shows the association between eachgraph shown in part (b) of FIG. 11 and class_idx.

(When Offset Type is 5 or 6 (Band Offset))

When the offset type supplied from the offset-type determining section623 is 5 or 6, the classifying section 624 classifies the pixel value ofthe subject pixel x as one of a plurality of classes in accordance withthe pixel value pic[x] of the subject pixel x.

-   -   When offset type is 5 (sao_type_idx=5)

When the pixel value pic[x] of the subject pixel x satisfies:(max×¼)≤pic[x]≤(max×¾), the classifying section 624 classifies thesubject pixel as a class other than 0. That is, when the pixel value ofthe subject pixel is contained within a range indicated by the hatchedportion in part (a) of FIG. 12, the classifying section 624 classifiesthe subject pixel as a class other than 0. In the above-describedcondition, max indicates the maximum value that can be taken by thesubject pixel x, and for example, max=255. When max=255, theabove-described condition may be represented by 8≤pic[x]/8≤23.

-   -   When offset type is 6 (sao_type_idx=6)

When the pixel value pic[x] of the subject pixel x satisfies:pic[x]≤(max×¼) or (max×¾)≤pic[x], the classifying section 624 classifiesthe subject pixel as a class other than 0. That is, when the pixel valueof the subject pixel is contained within a range indicated by thehatched portion in part (b) of FIG. 12, the classifying section 624classifies the subject pixel as a class other than 0. In theabove-described condition, max indicates the maximum value that can betaken by the subject pixel x, and for example, max=255. When max=255,the above-described condition may be represented by (pic[x]/8)≤7 or24≤(pic[x]/8).

The classifying processing performed by the classifying section 624 willbe more specifically described below.

When the offset type is 5 or 6, the classifying section 624 determinesthe class index (class_idx) of the class to which the subject pixel xbelongs as follows.class_idx=EoTbl[sao_type_idx][pic[x]/8]where EoTbl[sao_type_idx][pic[x]/8] is a transform table used fordetermining class_idx from the pixel value pic[x] of the subject pixel xand sao_type_idx. A specific example of the transform table EoTbl isshown in FIG. 12. In part (c) of FIG. 12, “BO_1” indicates thatsao_type_index=5, and “BO_2” indicates that sao_type_index=6.

As shown in part (c) of FIG. 12, when sao_type_index=5, the classifyingsection 624 classifies the subject pixel x as one of class indexes 1 to16 in accordance with the magnitude of pic[x] if the pixel value pic[x]of the subject pixel x satisfies the condition 8≤(pic[x]/8)≤23.

When sao_type_index=6, the classifying section 624 classifies thesubject pixel x as one of class indexes 1 to 16 in accordance with themagnitude of pic[x] if the pixel value pic[x] of the subject pixel xsatisfies the condition (pic[x]/8)≤7 or 24≤(pic[x]/8).

Generally, when the bit depth of an image is PIC_DEPTH, the maximumvalue of the subject pixel can be represented by max=2^(PIC_DEPTH)−1,and thus, classifying is performed by using pic/2^((PIC_DEPTH−5))instead of pic/8 in part (c) of FIG. 12.

(Second Example of Classifying Processing of Classifying Section 624)

A second example of classifying processing performed by the classifyingsection 624 will be described below.

In this example of the processing, the classifying section 624determines EdgeType by using the following mathematical expression (1-2)instead of mathematical expression (1-1). The other features are similarto those of the first example of classifying processing.EdgeType=Sign((pic[x]>>shift)−(pic[a]>>shift))+Sign((pic[x]>>shift)−(pic[b]>>shift))+2  (1−2)where “>>” denotes bitwise right shift, and “shift” denotes themagnitude of bit shift. The specific value of “shift” may be determinedsuch that it has a positive correlation with the bit depth of the pixelvalue.

In the classifying processing of the first example, even when thegradient of the pixel value is very small, if it is not 0, the value ofSign is not 0. Accordingly, in the classifying processing of the firstexample, the value of EdgeType is vulnerable to the influence of noise.

In this example of the processing, after performing bitwise right shifton the pixel value, the difference between the pixel values iscalculated. Thus, the value of EdgeType is less vulnerable to theinfluence of noise, thereby achieving the effect of improving the codingefficiency.

In this example of the processing, the following mathematical expression(1-3) may be used instead of mathematical expression (1-2).EdgeType=Sign((pic[x]−pic[a])>>shift)+Sign((pic[x]−pic[b])>>shift))+2  (1-3)

That is, after the difference between the pixel values is calculated,the bitwise right shift may be performed. By using mathematicalexpression (1-3), advantages similar to obtained by using mathematicalexpression (1-2) can be achieved.

(Third Example of Classifying Processing of Classifying Section 624)

A third example of classifying processing performed by the classifyingsection 624 will be described below.

In this example of the processing, the definition of the function Signdiscussed in the first example of the classifying processing calculatedby the classifying section 624 is changed as follows.

The other features are similar to those of the first example of theclassifying processing.Sign(z)=+1 (when z>th)Sign(z)=0 (when −th≤z≤th)Sign(z)=−1 (when z<−th)where th denotes a threshold having a predetermined value. The specificvalue of the threshold th may be determined such that the absolute valueof the threshold th has a positive correlation with the bit depth of thepixel value.

In this example of the processing, too, the value of EdgeType is lessvulnerable to the influence of noise, thereby achieving a high codingefficiency.

(Fourth Example of Classifying Processing of Classifying Section 624)

A fourth example of classifying processing performed by the classifyingsection 624 will be described below.

In this example of the processing, the classifying section 624 utilizesEoTbl[sao_type_idx][pic[x]/8] shown in FIG. 13 instead ofEoTbl[sao_type_idx][pic[x]/8] shown in part (c) of FIG. 12.

As shown in FIG. 13, in this example of the processing, when the valueof pic[x]/8 is 8 or 9, the subject pixel x is classified as a classhaving a class index other than 0, regardless of whethersao_type_index=5 or sao_type_index=6. Moreover, when the value ofpic[x]/8 is 22 or 23, the subject pixel x is classified as a classhaving a class index other than 0, regardless of whethersao_type_index=5 or sao_type_index=6.

In this example of the processing, when the pixel value of a subjectpixel is 15 or smaller (when pic[x]/8 is 0 or 1), the subject pixel isclipped to MIN. Moreover, when the pixel value of a subject pixel is 240or greater (when pic[x]/8 is 30 or 31), the subject pixel is clipped toMAX. As MIN and MAX, one of the following combinations is preferablyused.

-   -   MIN=15, MAX=240    -   MIN=16, MAX=239    -   MIN=16, MAX=235

In the first example of the classifying processing, a subject pixelwhich is classified as class 0 when sao_type_index=5 is classified as aclass other than 0 when sao_type_index=6. Moreover, a subject pixelwhich is classified as class 0 when sao_type_index=6 is classified as aclass other than 0 when sao_type_index=5.

Accordingly, in the first example of the classifying processing, thepixel value to which an offset is added may be very different dependingon whether sao_type_index=5 or sao_type_index=6, which may lead to aproblem in that the coding efficiency is not sufficiently improved asexpected. Such a problem may be noticeable when pic[x]/8 of the pixelvalue of an image which has not yet been subjected to offset filteringis 8 or 9 or 22 or 23.

In this example of the processing, when pic[x]/8 is 8 or 9, the subjectpixel x is classified as a class other than 0, regardless of whethersao_type_index=5 or sao_type_index=6. Moreover, when pic[x]/8 is 22 or23, the subject pixel x is classified as a class other than 0,regardless of whether sao_type_index=5 or sao_type_index=6. Accordingly,it is unlikely that the above-described problem will occur. Thus, byperforming processing of this example, the coding efficiency can beimproved.

In this example of the processing, in a case in which the value ofpic[x]/8 is 8 or 9 and in a case in which the value of pic[x]/8 is 22 or23, the subject pixel x is classified as a class other than 0,regardless of whether sao_type_index=5 or sao_type_index=6. However,this is not a limitation to implement this example of the processing. Inshort, processing is performed such that, when the value of pic[x]/8 iswithin a predetermined range, the subject pixel x is classified as aclass other than 0, regardless of whether sao_type_index=5 orsao_type_index=6.

Generally, when the pixel bit depth of an image is PIC_DEPTH, themaximum value of the subject pixel can be represented bymax=2^(PIC_DEPTH)−1, and thus, classifying is performed by usingpic/2^((PIC_DEPTH−5)) instead of pic/8 in FIG. 13.

In this manner, when the pixel value of the above-described subjectpixel is within a predetermined range, the classifying section 624 ofthis example may classify the subject pixel as an offset class in whichan offset will be added, regardless of whether the offset type of a unitarea containing the subject pixel is the above-described first offsettype or the above-described second offset type.

(Video Coding Device 2)

A description will now be given below, with reference to FIG. 14 throughparts (a) to (d) of FIG. 18, of the video coding device 2 whichgenerates coded data #1 by coding a subject image. The video codingdevice 2 partially includes a method defined in H. 264/MPEG-4. AVC, amethod used in KTA software, which is a joint development codec in VCEG(Video Coding Expert Group), a method used in TMuC (Test Model underConsideration) software, which is a successor codec to the codec used inKTA software, and a technology used in HM (HEVC TestModel) software.

FIG. 14 is a block diagram illustrating the configuration of the videocoding device 2 of this embodiment. The video coding device 2 includes,as shown in FIG. 14, a transform-and-quantize unit 21, a variable-lengthcode coder 22, an inverse-quantize-and-inverse-transform unit 23, abuffer memory 24, an intra-prediction image generator 25, aninter-prediction image generator 26, a motion-vector detector 27, aprediction-method controller 28, a motion-vector redundancy eliminatingunit 29, an adder 31, a subtractor 32, a deblocking filter 33, anadaptive filter 70, and an adaptive offset filter 80. The video codingdevice 2 codes a video image #10 (subject image) so as to generate codeddata #3.

The transform-and-quantize unit 21 performs: (1) DCT (Discrete CosineTransform) transform on each block of a prediction residual D obtainedby subtracting a prediction image Pred from a subject image, (2)quantizing a DCT coefficient obtained by performing DCT transform, and(3) supplying a quantized prediction residual QD obtained by performingquantization to the variable-length code coder 22 and theinverse-quantize-and-inverse-transform unit 23. Thetransform-and-quantize unit 21 also performs: (1) selecting aquantization step QP to be used for quantization for each tree block;(2) supplying a quantization parameter difference Δqp representing thelevel of the selected quantization step QP to the variable-length codecoder 22; and (3) supplying the selected quantization step QP to theinverse-quantize-and-inverse-transform unit 23. The quantizationparameter difference Δqp is a difference value obtained by subtractingthe value of the quantization parameter qp′ concerning a tree blockwhich has been subjected to DCT-transform and quantization immediatelybefore a subject tree block from the quantization parameter qp(QP=−2^(pq/6)) concerning the subject tree block to be subjected toDCT-transform and quantization.

The variable-length code coder 22 generates coded data #1 by performingvariable-length coding on: (1) the quantized prediction residual QD andΔqp supplied from the transform-and-quantize unit 21; (2) a predictionparameter PP supplied from the prediction-method controller 28, whichwill be discussed later, and (3) a filter set number, a filtercoefficient group, area specifying information, and ON/OFF informationsupplied from the adaptive filter 70, which will be discussed later. Thevariable-length code coder 22 also codes QAOU information supplied fromthe adaptive offset filter 80 and inserts the coded QAOU informationinto the coded data #3.

The inverse-quantize-and-inverse-transform unit 23 performs: (1)inverse-quantizing the quantized prediction residual QD, (2) inverse-DCT(Discrete Cosine Transform) on a DCT coefficient obtained by performinginverse quantization, and (3) supplying a prediction residual D obtainedby performing inverse DCT to the adder 31. When inverse-quantizing thequantized prediction residual QD, the quantization step QP supplied fromthe transform-and-quantize unit 21 is utilized. The prediction residualD output from the inverse-quantize-and-inverse-transform unit 23 is theprediction residual D input into the transform-and-quantize unit 21 towhich a quantization error is added, however, for a simple description,the same name will be used.

The intra-prediction image generator 25 generates a prediction imagePred_Intra concerning each partition. More specifically, theintra-prediction image generator 25 performs: (1) selecting a predictionmode concerning each partition used for intra prediction; and (2)generating the prediction image Pred_Intra from the decoded image P byusing the selected prediction mode. The intra-prediction image generator25 supplies the generated prediction image Pred_Intra to theprediction-method controller 28.

The intra-prediction image generator 25 also specifies a predictionindex PI concerning each partition from a selected prediction modeselected for the partition and from the size of the partition, and thensupplied the prediction index PI to the prediction-method controller 28.

The intra-prediction image generator 25 also supplies the size of asubject partition and intra-prediction mode information IEM, which isinformation indicating a prediction mode assigned to the subjectpartition, to the adaptive filter 70.

The motion-vector detector 27 detects a motion vector my concerning eachpartition. More specifically, the motion-vector detector 27 detects amotion vector my concerning a subject partition by: (1) selecting afiltered decoded image P_FL′ to be used as a reference image; and (2) bysearching for an area which best approximates to the subject partitionin the selected filtered decoded image P_FL′. The filtered decoded imageP_FL′ is an image obtained by performing deblocking processing by usingthe deblocking filter 33, adaptive offset processing by using theadaptive offset filter 80, and adaptive filtering processing by usingthe adaptive filter 70 on a decoded image. The motion-vector detector 27may read pixel values of individual pixels forming the filtered decodedimage P_FL′ from the buffer memory 24. The motion-vector detector 27supplies the detected motion vector my to the inter-prediction imagegenerator 26 and the motion-vector redundancy eliminating unit 29,together with a reference image index RI which specifies the filtereddecoded image P_FL′ utilized as a reference image. Concerning apartition on which bidirectional prediction (weighted prediction) willbe performed, the motion-vector detector 27 selects two filtered decodedimages P_FL′ and P_FL2′ as reference images, and supplies motion vectorsmv1 and mv2 and reference image indexes RI1 and RI2 associated with thetwo filtered decoded images P_FL1′ and P_FL2′ to the inter-predictionimage generator 26 and the motion-vector redundancy eliminating unit 29.

The inter-prediction image generator 26 generates a motion-compensatedimage me concerning each inter-prediction partition. More specifically,the inter-prediction image generator 26 generates a motion-compensatedimage me from the filtered decoded image P_FL′ specified by thereference image index RI supplied from the motion-vector detector 27 byusing the motion vector my supplied from the motion-vector detector 27.As in the motion-vector detector 27, the inter-prediction imagegenerator 26 may read pixel values of individual pixels forming thefiltered decoded image P_FL′ from the buffer memory 24. Theinter-prediction image generator 26 supplies the generatedmotion-compensated image me (inter-prediction image Pred_Inter) to theprediction-method controller 28, together with the reference image indexRI supplied from the motion-vector detector 27. Concerning a partitionon which bidirectional prediction (weighted prediction) will beperformed, the inter-prediction image generator 26 performs: (1)generating a motion-compensated image mc1 from a filtered decoded imageP_FL′ specified by the reference image index RI1 by using a motionvector mv1; (2) generating a motion-compensated image mc2 from afiltered decoded image P_FL2′ specified by the reference image index RI2by using a motion vector mv2; and (3) generating an inter-predictionimage Pred_Inter by adding an offset value to a weighted average of themotion-compensated image mc1 and the motion-compensated image mc2.

The prediction-method controller 28 compares each of theintra-prediction image Pred_Intra and the inter-prediction imagePred_Inter with a subject image to be coded, and determines whether toperform intra prediction or inter prediction. If intra prediction isselected, the prediction-method controller 28 supplies theintra-prediction image Pred_Intra to the adder 31 and the subtractor 32as the prediction image Pred, and also supplies the prediction index PIsupplied from the intra-prediction image generator 25 to thevariable-length code coder 22 as the prediction parameter PP. Incontrast, if inter prediction is selected, the prediction-methodcontroller 28 supplies the inter-prediction image Pred_Inter to theadder 31 and the subtractor 32 as the prediction image Pred, and alsosupplies the reference image index RI supplied from the inter-predictionimage generator 26 and an estimation motion-vector index PMVI andmotion-vector residual MVD supplied from the motion-vector redundancyeliminating unit 29 (discussed later) to the variable-length code coderas the prediction parameters PP.

By subtracting the prediction image Pred selected in theprediction-method controller 28 from the subject image to be coded, aprediction residual D is generated in the subtractor 32. The predictionresidual D generated in the subtractor 32 is subjected to DCT-transformand quantization by the transform-and-quantize unit 21, as stated above.Meanwhile, by adding the prediction image Pred selected in theprediction-method controller 28 to the prediction residual D generatedin the inverse-quantize-and-inverse-transform unit 23, a locally decodedimage P is generated in the adder 31. The locally decoded image Pgenerated in the adder 31 passes through the deblocking filter 33, theadaptive offset filter 80, and the adaptive filter 70. Then, the locallydecoded image P is stored in the buffer memory 24 as a filtered decodedimage P_FL and is utilized as a reference image for performing interprediction.

The motion-vector redundancy eliminating unit 29 eliminates a redundancyin a motion vector my detected by the motion-vector detector 27. Morespecifically, the motion-vector redundancy eliminating unit 29 performs:(1) selecting an estimation method used for estimating the motion vectormv; (2) determining an estimation motion vector pmv in accordance withthe selected estimation method; and (3) generating a motion-vectorresidual MVD by subtracting the estimation motion vector pmv from themotion vector my. The motion-vector redundancy eliminating unit 29supplies the generated motion-vector residual MVD to theprediction-method controller 28, together with the estimationmotion-vector index PMVI indicating the selected estimation method.

When the difference between the pixel values of pixels adjacent to eachother with a block boundary or a CU boundary therebetween in the decodedimage P is smaller than a predetermined threshold, the deblocking filter33 performs deblocking processing on the block boundary or the CUboundary in the decoded image P, thereby smoothing an image in thevicinity of the block boundary or the CU boundary. The image subjectedto deblocking processing by the deblocking filter 33 is output to theadaptive offset filter 80 as a deblocked decoded image P_DB.

The adaptive offset filter 80 performs adaptive offset filteringprocessing on the deblocked decoded image P_DB supplied from thedeblocking filter 33, thereby generating an offset-filtered decodedimage P_OF. The generated offset-filtered decoded image P_OF is suppliedto the adaptive filter 70. The specific configuration of the adaptiveoffset filter 80 will be discussed later, and thus, an explanationthereof is omitted here.

The adaptive filter 70 performs adaptive filtering processing on theoffset-filtered decoded image P_OF supplied from the adaptive offsetfilter 80, thereby generating a filtered decoded image P_FL. Thefiltered decoded image P_FL subjected to filtering processing by theadaptive filter 70 is stored in the buffer memory 24. A filtercoefficient used by the adaptive filter 70 is determined so that theerror between the filtered decoded image P_FL and the subject image #10can be minimized, and the filter coefficient determined in this manneris coded as a filter parameter FP and is transmitted to the videodecoding device 1.

(Adaptive Offset Filter 80)

FIG. 15 is a block diagram illustrating the configuration of theadaptive offset filter 80. The adaptive offset filter 80 includes, asshown in FIG. 15, an adaptive offset filter information setting unit 81and an adaptive offset filter processor 82.

The adaptive offset filter information setting unit 81 includes, asshown in FIG. 15, an offset calculator 811, an offset shifter 816, anoffset clipping section 812, an offset information selector 813, and anoffset attribute setting section 815.

(Offset Calculator 811)

The offset calculator 811 calculates offsets concerning all the offsettypes and all the classes for all QAOMUs up to a predetermined splitdepth included in the unit of processing (for example, an LCU). In thiscase, the offset types and the classes are the same as those discussedin a description of the video decoding device 1.

FIG. 16 is a flowchart of a flow of processing performed by the offsetcalculator 811.

(Step S201)

First, the offset calculator 811 starts a first loop using the QAOMUnumber of a subject QAOMU as a loop variable. For example, in theexample shown in parts (a) through (e) of FIG. 7, the first loop is aloop from the QAOMU number=0 to the QAOMU number=340.

(Step S202)

Then, the offset calculator 811 starts a second loop using the offsettype that can be selected for the subject QAOMU as a loop variable. Thesecond loop is a loop from offset type 1 to offset type 6.

(Step S203)

Then, the offset calculator 811 starts a third loop using a pixelincluded in the subject QAOMU as a unit.

(Step S204)

Then, the offset calculator 811 classifies a subject pixel under one ofa plurality of classes. More specifically, when the offset type, whichis the loop variable of the second loop, is one of 1 to 4, the offsetcalculator 811 classifies the subject pixel under one of class 1 throughclass 4. The classifying processing in this step is the same processingas the classifying processing in one of the first through fourthexamples performed by the classifying section 624 of the adaptive offsetfilter 60 of the video decoding device 1.

Concerning the subject QAOMU, the offset calculator 811 also calculatesthe classify number count[part_idx][sao_type_index][class_idx], which isthe number of times pixels are classified for each class. In thisclassify number, part_idx indicates the QAOMU number.

(Step S205)

Then, the offset calculator 811 calculates a difference pixel value forthe subject pixel by calculating the difference between the pixel valueof the subject pixel in the deblocked decoded image P_DB and the pixelvalue of the subject pixel in the subject image #10 to be coded. Morespecifically, when the position of the subject pixel is represented by(x, y), the offset calculator 811 calculates P_DB(x, y)−Org(x, y). Inthis expression, P_DB(x, y) denotes the pixel value of the subject pixelin the deblocked decoded image P_DB, and Org(x, y) denotes the pixelvalue of the subject pixel in the subject image #10.

(Step S206)

This step is the termination of the third loop. At the time point atwhich this step has been completed, difference pixel values have beencalculated for all the pixels included in the subject QAOMU.

(Step S207)

Then, the offset calculator 811 calculates an offset by dividing the sumof the difference pixel values of the pixels included in the subjectQAOMU for each class by the above-described classify number concerningthis class. More specifically, the offset calculator 811 calculates anoffset offset[part_idx][sao_type_idx][class_idx] for the subject QAOMU,the subject offset type, and the subject class by using the followingequation.offset[part_idx][sao_type_idx][class_idx]=Σ(P_DB(x,y)−Org(x,y))/count[part_idx][sao_type_idx][class_idx],where the symbol Σ denotes the sum of the pixels classified as thesubject class specified by class_idx in the subject QAOMU specified bypart_idx and the subject offset type specified by sao_type_idx.(Step S208)

This step is the termination of the first loop.

(Step S209)

This step is the termination of the first loop.

By performing the above-described processing, the offset calculator 811calculates offsets concerning all the offset types and all the classesfor all QAOMUs up to a predetermined split depth included in a subjectLCU. For example, in the case of the example shown in parts (a) through(e) of FIG. 7, the offset calculator 811 calculates a total of 16368offsets as represented as follows:((the total number of QAOMUs in the split depth 0)+ . . . +(the totalnumber of QAOMUs in the split depth 4))×(the number of EO offsettypes)×(the number of classes of EO)+(the number of BO offsettypes)×(the number of classes ofBO))=(1+4+16464+256)×((4×4)+(2×16))=16368.The number of bits of each offset is, for example, ten bits.

The offset calculator 811 supplies offset information indicating offsetscalculated by the above-described processing, offset types, classes, andQAOU structure information indicating the QAOU split structure to theoffset shifter 816.

In step S204, the video coding device 2 may code a flag indicating whichtype of classifying processing has been performed, and then, theadaptive offset filter 60 of the video decoding device 1 may refer tothis flag and may perform the same classifying processing as thatindicated by the flag. Alternatively, without using such a flag, thesame classifying processing which has been determined in the videocoding device 2 and the video decoding device 1 in advance may beperformed.

(Offset Shifter 816)

The offset shifter 816 quantizes each of the offsets included in theoffset information supplied from the offset calculator 811. The offsetshifter 816 quantizes the offsets by performing bitwise right shift onthe offsets so that the precision of the offsets may be transformed fromthe pixel bit depth to the offset bit depth. The amount by which anoffset is shifted in the shifting processing is determined by the shiftvalue supplied from the offset attribute setting section 815, which willbe discussed later.

(Offset Clipping Section 812)

In order to restrict the offsets to the offset value range supplied fromthe offset attribute setting section 815, which will be discussed later,the offset clipping section 812 performs clip processing by using one ofthe following first clip processing and second clip processing on theoffsets supplied from the offset shifter 816.

(First Clip Processing)

The offset clipping section 812 performs clip processing on each of theoffsets included in the offset information supplied from the offsetshifter 816. The offset clipping section 812 clips each offset suppliedfrom the offset shifter 816 to, for example, values from −8 to 7,thereby expressing each offset by four bits. The clipped offsets aresupplied to the offset information selector 813. The bit width used forclipping is set in accordance with the image bit depth and the offsetbit depth, as in the video decoding device 1.

By clipping each offset in this manner, the memory size of a memory (notshown) to store each offset can be reduced. The amount of data requiredto code offsets included in the coded data #1 can also be decreased,thereby making it possible to improve the coding efficiency.Additionally, since excessive offsets are not added, a suitable level ofimage quality can be guaranteed.

(Second Clip Processing)

The offset clipping section 812 may change the value for the clippingrange of an offset supplied from the offset shifter 816 in accordancewith the offset type.

For example, if the offset type is an edge offset, the number of bits ofan offset is set to be eight bits, and if the offset type is a bandoffset, the number of bits of an offset is set to be four bits. Moregenerally, when the number of bits of an edge-offset offset is N bitsand the number of bits of a band-offset offset is M bits, the numbers ofbits of offsets are determined so that the condition N>M can besatisfied.

In this manner, by varying the number of bits of an offset in accordancewith the offset type, the coding efficiency can be improved withoutrequiring an excessive memory size of a memory for storing each offset.

When the threshold th for restricting values that can be taken as anoffset is greater than 2^(m-1) but not greater than 2^(m), m-bitfixed-length coding method can be used as the coding method for codingthe offset. More specifically, Truncated unary coding or Truncated Ricecoding having th as the maximum value may be used. The above-describedmaximum value th is determined by the offset value range supplied fromthe offset attribute setting section 815.

Clip processing performed by a combination of the above-described firstclip processing and second clip processing is also included in thisembodiment. The adaptive offset filter 80 may not include the offsetclipping section 812.

(Offset Information Selector 813)

The offset information selector 813 determines a combination of anoffset type, a class, and an offset which minimize the RD cost(Rate-Distortion cost) and the associated QAOU split structure, andsupplies QAOU information indicating the determined offset types,classes, and offsets, and the associated QAOM split structure to thevariable-length code coder 22. The offset information selector 813 alsosupplies the determined offsets for each QAOU or each QAOMU to theadaptive offset filter processor 82.

Processing performed by the offset information selector 813 will bediscussed more specifically with reference to FIGS. 17 and 18. FIG. 17is a flowchart of a flow of processing performed by the offsetinformation selector 813.

(Step S301)

The offset information selector 813 starts a first loop using the QAOMUnumber of a subject QAOMU as a loop variable.

(Step S302)

Then, the offset information selector 813 starts a second loop using theoffset type that can be selected for the subject QAOMU as a loopvariable. The second loop is a loop from offset type 1 to offset type 6.

(Step S303)

Then, the offset information selector 813 calculates, for a subjectoffset type, the squared error between the offset-filtered decoded imageP_OF and the subject coded image #10 concerning the subject QAOMU.

(Step S304)

This step is the termination of the second loop.

(Step S305)

This step is the termination of the first loop. At the time point atwhich the first loop and the second loop have been completed, thesquared errors concerning each QAOMU for all the offset types have beencalculated.

(Step S306)

Then, the offset information selector 813 determines, among QAOU splitstructures which may be obtained by dividing the subject unit ofprocessing (for example, an LCU) into QAOUs, the QAOU split structurethat minimizes the RD cost.

Specific examples of the processing performed by the offset informationselector 813 in this step will be described below with reference toparts (a) through (e) of FIG. 18.

First, the offset information selector 813 calculates the RD cost whenthe split depth is 0 and the RD cost when the split depth is 1 (part (a)and (b) of FIG. 18). In part (c) of FIG. 18, it is assumed that the RDcost of the split depth 1 is smaller than that of the split depth 0.

Then, the offset information selector 813 calculates the RD cost whenthe split depth is 2 (part (d) of FIG. 18).

Then, the offset information selector 813 compares the RD cost of aQAOMU of the split depth 1 with that of QAOMUs of the split depth 2contained in the QAOMU of the split depth 1. If the RD cost of theQAOMUs of the split depth 2 is smaller than that of the QAOMU of thesplit depth 1, the offset information selector 813 updates the QAOMU ofthe split depth 1 to the QAOMUs of the split depth 2 (part (e) of FIG.18). The offset information selector 813 repeats this processing up tothe largest split depth. In this manner, the QAOU split structure whichminimizes the RD Cost can be determined.

(Offset Attribute Setting Section 815)

The offset attribute setting section 815 receives the pixel bit depth(not shown) and determines the offset bit depth. The offset attributesetting section 815 then sets the offset value range and the shift valueby using the determined offset bit depth. The offset value range issupplied to the adaptive offset filter processor 82, and the shift valueis supplied to the offset shifter 816. The setting of the offset valuerange and the shift value are the same processing as that performed bythe above-described offset attribute setting section 613, and thus, anexplanation thereof is omitted here.

(Adaptive Offset Filter Processor 82)

The adaptive offset filter processor 82 adds an offset supplied from theoffset information selector 813 to each pixel of a subject QAOU in thedeblocked decoded image P_DB. The adaptive offset filter processor 82outputs, as an offset-filtered decoded image P_OF, an image obtained byperforming processing on all the QAOUs included in the deblocked decodedimage P_DB. The configuration of the adaptive offset filter processor 82is the same as that of the adaptive offset filter processor 62, andthus, an explanation thereof is omitted here. Each offset stored in anoffset information storage section (not shown) included in the adaptiveoffset filter processor 82 is restricted to the offset value range setby the offset attribute setting section 815.

Second Embodiment

In the first embodiment, sao_offset[sao_curr_depth][ys][xs][i] includedin the coded data #1 is syntax which represents a specific value of anoffset to be added to each pixel included in a subject QAOU in theoffset filtering processing performed by using the adaptive offsetfilter.

Meanwhile, the present inventors have found that the amount of datarequired for coding data can be further reduced by performing predictivecoding on an offset value used for offset filtering processing, that is,by coding an offset residual calculated by using an offset value and aprediction value of the offset value.

In a second embodiment, a description will be given, with reference toFIGS. 19 through 21, of a video decoding device which decodes an offsetsubjected to predictive coding and which performs offset filteringprocessing, and of a video coding device which performs predictivecoding on an offset used for offset filtering processing. Portionsdiscussed in the first embodiment will not be described.

(Coded Data)

Coded data of this embodiment includes an offset residualsao_offset_residual[sao_curr_depth][ys][xs][i] instead ofsao_offset[sao_curr_depth][ys][xs][i] included in the coded data #1 ofthe first embodiment. The other portions of the configuration of thecoded data of this embodiment are similar to those of the coded data #1of the first embodiment. Hereinafter, the coded data of this embodimentmay also be indicated as coded data #3.

(sao_offset_residual)

The offset residual sao_offset residual[sao_curr_depth][ys][xs][i]represents a weighted difference value between an offset value to beadded to each pixel included in a QAOU in offset filtering processingperformed by an adaptive offset filter of this embodiment and aprediction value of the offset value, and is also indicated bysao_offset_residual[sao_type_idx][class_idx].

If an offset to be added to a subject pixel included in a subject QAOUis indicated by Offset[sao_type_idx][class_idx], the offset residualsao_offset_residual[sao_type_idx][class_idx] is represented as follows:sao_offset_residual[sao_type_idx][class_idx]=Offset[sao_type_idx][class_idx]−a*pred_offset[merge_tbl[sao_type_idx]][class_idx]where a is a weight coefficient to be multiplied by the prediction valuepred_offset and merge_tbl is a function using sao_type_idx as anargument. Specific examples of a and merge_tbl will be discussed later,and thus, an explanation thereof is omitted here.(Video Decoding Device)

The video decoding device of this embodiment includes an adaptive offsetfilter 60′ instead of the adaptive offset filter 60 of the videodecoding device 1 of the first embodiment. The other elements of theconfiguration of the video decoding device of this embodiment aresimilar to those of the video decoding device 1 of the first embodiment.

FIG. 19 is a block diagram illustrating the configuration of theadaptive offset filter 60′ of this embodiment. As shown in FIG. 19, theadaptive offset filter 60′ includes an offset information decodingsection 611′ instead of the offset information decoding section 611 ofthe adaptive offset filter 60.

(Offset Information Decoding Section 611)

The offset information decoding section 611′ refers to QAOU informationincluded in the coded data #3 and decodes offset information OI includedin the QAOU information. By using an offset residualsao_offset_residual[sao_type_idx][class_idx] and a prediction valuepred_offset[merge_tbl[sao_type_idx]][class_idx] obtained by decoding theoffset information OI, the offset information decoding section 611′calculates an offset Offset[sao_type_idx][class_idx] to be used foradaptive offset filtering processing by the following expression:Offset[sao_type_idx][class_idx]=a*pred_offset[merge_tbl[sao_type_idx]][class_idx]+sao_offset_residual[sao_type_idx][class_idx].The offset information decoding section 611′ then stores the calculatedOffset[sao_type_idx][class_idx] in the offset information storagesection 621. In the above-described expression,pred_offset[merge_tbl[sao_type_idx]][class_idx] is a prediction value ofOffset[sao_type_idx][class_idx], and merge_tbl[sao_type_idx] representsa table for providing an index to sao_type_idx=1 to 6, and one or moreitems of sao_type_idx may be considered as the same group.(First Specific Example of pred_offset)

A first specific example ofpred_offset[merge_tbl[sao_type_idx]][class_idx] will be discussed below.In this example, the prediction valuepred_offset[merge_tbl[sao_type_idx]][class_idx] is determined by:pred_offset[merge_tbl[sao_type_idx]][class_idx]=Offset′[sao_type_idx][class_idx]where Offset′[sao_type_idx][class_idx] is a decoded offset, andrepresents an offset associated with the offset type index sao_type_idxand the class index class_idx.

In this manner, in this example, as the prediction value ofOffset[sao_type_idx][class_idx], an offsetOffset′[sao_type_idx][class_idx], which is a decoded offset associatedwith the offset type index sao_type_idx and the class index class_idx isused.

(Second Specific Example of pred_offset)

A second specific example ofpred_offset[merge_tbl[sao_type_idx]][class_idx] will be discussed below.In this example, the prediction valuepred_offset[merge_tbl[sao_type_idx]][class_idx] is determined by:pred_offset[merge_tbl[sao_type_idx]][class_idx]=(pred_offset′[merge_tbl[sao_type_idx]][class_idx]*W1+Offset′[sao_type_idx][class_idx]*W2>>log₂(W1+W2)where pred_offset′[merge_tbl[sao_type_idx]][class_idx] denotes aprediction value used for calculating a decoded offsetOffset′[sao_type_idx][class_idx], “*” denotes an operation symbolrepresenting multiplication, and “>>” denotes a bitwise right shift. W1and W2 denote weight coefficients, and may take values, such as W1=3 andW2=1. Specific values of W1 and W2 may be determined so that the codingefficiency can be increased.

As is seen from the above-described expression, in this example,reference is made to decoded prediction values and offsets recursively,such as reference is made to pred_offset′ and Offset′ in order todetermine pred_offset, reference is made to pred_offset″ and Offset″ inorder to determine pred_offset′, and so on. Accordingly, a plurality ofdecoded offsets contribute to pred_offset, thereby suppressing excessivefluctuations in a prediction value. With this arrangement, for example,even if an inappropriate prediction value has been calculated due to theinfluence of noise, the influence of such an inappropriate predictionvalue can be suppressed, thereby making it possible to improve thecoding efficiency.

(Third Specific Example of pred_offset)

A third specific example ofpred_offset[merge_tbl[sao_type_idx]][class_idx] will be discussed below.In this example, the prediction valuepred_offset[merge_tbl[sao_type_idx]][class_idx] is determined by:pred_offset[merge_tbl[sao_type_idx]][class_idx]=clip3(−th,th,pred_offset[merge_tbl[sao_type_idx]][class_idx])where clip3(A, B, C) denotes that the value C is clipped by the lowerlimit value A and the upper limit value B. The argumentpred_offset[merge_tbl[sao_type_idx]] of clip3 is determined, forexample, as in the above-described first or second specific example. Thethreshold th is determined as follows while being dependent on the bitdepth bit_depth of the pixel value.th=4 (bit_depth=8)th=8 (bit_depth>8)

In this manner, in this example, by using a prediction value clipped bythe upper limit value and the lower limit value, excessively largeprediction values and excessively small prediction values are notgenerated, thereby making it possible to improve the coding efficiency.The absolute values of the upper limit value and the lower limit valueare set such that they become large when the bit of a pixel value islarge. Thus, appropriate clip processing is performed in accordance withthe bit depth of a pixel value, thereby preventing degradation of theimage quality.

(First Specific Example of merge_tbl)

Part (a) of FIG. 20 is a table illustrating a first specific example ofthe function merge_tbl[sao_type_idx]. As shown in part (a) of FIG. 20,when sao_type_idx=0, merge_tbl[sao_type_idx] of this example does nottake any value, and when sao_type_idx=1 to 6, merge_tbl[sao_type_idx] ofthis example takes 0 to 5, respectively. Accordingly,merge_tbl[sao_type_idx] of this example may also be represented as:merge_tbl[sao_type_idx]=sao_type_idx−1.

By using merge_tbl[sao_type_idx] of this example, the offset informationdecoding section 611′ individually determines a prediction valuepred_offset for each sao_type_idx and for each class_idx. It is thuspossible to reduce the amount of data required to code offset residualssao_offset residual.

(Second Specific Example of merge_tbl)

Part (b) of FIG. 20 is a table illustrating a second specific example ofthe function merge_tbl[sao_type_idx]. As shown in part (b) of FIG. 20,in the case of an edge offset (sao_type_idx=1 to 4),merge_tbl[sao_type_idx] of this example takes 0, and in the case of aband offset (sao_type_idx=5 or 6), merge_tbl[sao_type_idx] of thisexample takes 1 or 2, respectively.

For example, if sao_type_idx=1 (merge_tbl[sao_type_idx=1]=0) andclass_idx=1 have been specified when calculating the previous offset,and if sao_type_idx=2 (merge_tbl[sao_type_idx=2]=0) and class_idx=1 havebeen specified when calculating the subsequent offset, the predictionvalue to be used for calculating the subsequent offset is the same asthat used for calculating the previous offset.

By using merge_tbl[sao_type_idx] of this example, the offset informationdecoding section 611′ performs the following processing depending onwhether an edge offset is specified or a band offset is specified.

-   -   When edge offset is specified

The prediction value of an offset to be decoded is calculated from adecoded offset classified as the same class as that of the offset to bedecoded. In this case, the offset type of the offset to be decoded andthat of the prediction value may be different as long as the class isthe same. Accordingly, a prediction value which is set for calculatingan offset of a certain offset type may be used for calculating an offsetof an offset type different from this certain offset type. As a result,the amount of processing for setting prediction values can be reduced.

-   -   When band offset is specified

The prediction value of an offset to be decoded is calculated from anoffset having the same offset type and classified as the same class asthose of the offset to be decoded.

By using merge_tbl[sao_type_idx] of this example, an appropriateprediction value can be calculated while the amount of processing isreduced.

(Specific Example of Coefficient a)

The weight coefficient a to be multiplied by the prediction valuepred_offset may be 1 regardless of the offset type, or may varydepending on the offset type.

For example, the weight coefficient a may vary as:

a=1 (in the case of an edge offset)

a=0.5 (in the case of a band offset).

More generally, if the coefficient a to be used when an edge offset isspecified is indicated by a(edge) and if the coefficient a to be usedwhen a band offset is specified is indicated by a(band), the coefficienta which satisfies the following condition is used:a(edge)>a(band).

The present inventors have found that the correlation between a decodedoffset and an offset to be decoded when an edge offset is specified isgreater than that when a band offset is specified. In theabove-described specific example, the influence of the correlationbetween a decoded offset and an offset to be decoded can beappropriately reflected, thereby reducing the amount of data to codeoffset residuals.

(Video Coding Device)

The video coding device of this embodiment includes an adaptive offsetfilter 80′ instead of the adaptive offset filter 80 of the video codingdevice 2 of the first embodiment. The other elements of theconfiguration of the video coding device of this embodiment are similarto those of the video coding device 2 of the first embodiment.

FIG. 21 is a block diagram illustrating the configuration of theadaptive offset filter 80′ of this embodiment. As shown in FIG. 21, theadaptive offset filter 80′ includes an offset residual determiningsection 814 in addition to the components of the adaptive offset filter80.

(Offset Residual Determining Section 814)

The offset residual determining section 814 calculates an offsetresidual by taking the difference between an offset supplied from theoffset information selector 813 and a prediction value of the offset.The offset residual is coded by the variable-length code coder 22 aspart of QAOU information.

The prediction value set by the offset residual determining section 814is similar to that set by the offset information decoding section 611′of the video decoding device of this embodiment, and thus, anexplanation thereof is omitted here.

(Appendix 1)

The present invention may be described as follows.

An image filtering device according to the present invention is an imagefiltering device for adding an offset to a pixel value of each pixelforming an input image which is constituted by a plurality of unitareas. The image filtering device includes: offset attribute settingmeans for setting offset attributes for a subject unit area by referringto offset-type specifying information included in coded data; offsetdecoding means for decoding an offset having a bit width correspondingto an offset value range included in the set offset attributes; andfiltering means for adding the offset to the pixel value of each pixelforming the input image.

With the image filtering device configured as described above, theoffset attribute setting means refers to the offset-type specifyinginformation included in the coded data and sets offset attributes forthe subject unit area, and the offset decoding means decodes an offsethaving a bit width corresponding to an offset value range included inthe set offset attributes. It is thus possible to effectively reduce thememory size of a memory for storing offsets.

Accordingly, with the above-described configuration, it is possible toperform appropriate offset filtering processing while the memory size ofa memory for storing offsets is reduced.

The offset-type specifying information may be determined for each of theinput images or for each of the unit areas. Alternatively, theoffset-type specifying information may be determined for eachpredetermined set of the input images or for each predetermined set ofthe unit areas.

The offset-type specifying information may preferably includeinformation concerning a bit depth of the pixel value in each unit areaforming the input image, and the offset decoding means may preferablydecode an offset having a bit width corresponding to the bit depth ofthe pixel value.

With the above-described configuration, since the offset decoding meansdecodes an offset having a bit width corresponding to the bit depth ofthe pixel value, the memory size of a memory for storing offsets caneffectively be reduced.

The offset-type specifying information may include informationconcerning a bit depth of the pixel value in each unit area forming theinput image, and the offset decoding means may decode an offset having abit width which can express a value range corresponding to the bitdepth.

With the above-described configuration, since the offset decoding meansdecodes an offset having a bit width which can express a value rangecorresponding to the pixel value in each unit area forming the inputimage, the memory size of a memory for storing offsets can be reducedeffectively.

The bit width that can express a value range corresponding to the bitdepth is a bit width when values included in the value range arerepresented in binary notation. For example, if the value range is −2³to 2³−1, the bit width that can express the value range is four bits.

The above-described decoded offset may preferably be a quantized value,and the filtering means may preferably add a value obtained byinverse-quantizing the offset by using a parameter included in theoffset attributes to the pixel value of each pixel.

With the above-described configuration, the decoded offset is aquantized value, and the filtering means adds a value obtained byinverse-quantizing the offset by using a parameter included in theoffset attributes to the pixel value of each pixel. Accordingly, anoffset corresponding to a parameter included in the offset attributes isadded to each image value.

Accordingly, with the above-described configuration, it is possible toimprove the coding efficiency while the memory size of a memory forstoring offsets is reduced.

The offset-type specifying information may include informationconcerning a shift value of the pixel value, and the filtering means mayadd a value obtained by inverse-quantizing the offset by using the shiftvalue instead of adding the above-described offset.

With the above-described configuration, the offset-type specifyinginformation includes information concerning a shift value of the pixelvalue, and the filtering means adds a value obtained byinverse-quantizing the offset by using the shift value instead of addingthe above-described offset, thereby making it possible to obtain anoffset corresponding to the shift value of the pixel value. Thus, it ispossible to improve the coding efficiency while the memory size of amemory for storing offsets is reduced.

The shift value of the pixel value indicates a difference value betweenthe pixel bit depth and the offset bit depth, and inverse-quantizing ofthe offset by using the shift value means that the offset is subjectedto bitwise left shift by an amount equal to the shift value, therebyperforming transformation from the offset bit depth to the pixel bitdepth.

The offset-type specifying information may preferably be determined forthe input image.

With the above-described configuration, since the offset-type specifyinginformation is determined for the input image, the image filteringdevice is capable of performing appropriate offset processing for theinput image.

An offset decoding device according to the present invention is anoffset decoding device for decoding each offset which is referred to byan image filter for adding an offset to a pixel value of each pixelforming an input image. The offset decoding device includes: offsetresidual decoding means for decoding each offset residual from codeddata; prediction value determining means for determining a predictionvalue of each offset from a decoded offset; and offset calculating meansfor calculating each offset from a prediction value determined by theprediction value determining means and an offset residual decoded by theoffset residual decoding means.

In the offset decoding device configured as described above, there areprovided the offset residual decoding means for decoding each offsetresidual from coded data, the prediction value determining means fordetermining a prediction value of each offset from a decoded offset, andthe offset calculating means for calculating each offset from aprediction value determined by the prediction value determining meansand an offset residual decoded by the offset residual decoding means.Accordingly, an offset can be appropriately decoded from coded datahaving a smaller amount of data, compared with a case in which eachoffset itself is coded.

The input image may preferably be constituted by a plurality of unitareas. The offset residual decoding means may preferably decode eachoffset residual in association with an offset type, which is determinedfor each unit area, and an offset class, which is determined for eachpixel. The prediction value determining means may preferably determine aprediction value of each offset from a decoded offset associated withthe same offset type and the same offset class as those of the offsetfor which a prediction value will be determined.

With the above-described configuration, the prediction value of eachoffset is determined from a decoded offset associated with the sameoffset type and the same offset class as those of the offset for which aprediction value will be determined, thereby making it possible toimprove the prediction precision. Accordingly, with the above-describedconfiguration, it is possible to appropriately decode an offset fromcoded data having a smaller amount of data.

The input image may preferably be constituted by a plurality of unitareas. The offset residual decoding means may preferably decode eachoffset residual in association with an offset type, which is determinedfor each unit area, and an offset class, which is determined for eachpixel. The prediction value determining means may preferably determine aprediction value of each offset from a decoded offset associated withthe same first offset type group and the same offset class as those ofthe offset for which a prediction value will be determined in a case inwhich the offset type associated with the offset belongs to a firstoffset type group, and the prediction value determining means maypreferably determine a prediction value of each offset from a decodedoffset associated with the same offset type and the same offset class asthose of the offset for which a prediction value will be determined in acase in which the offset type associated with the offset belongs to asecond offset type group.

With the above-described configuration, the prediction value of eachoffset is determined from a decoded offset associated with the sameoffset type as that of the offset for which a prediction value will bedetermined in a case in which the offset type associated with the offsetbelongs to a first offset type group, and the prediction value of eachoffset is determined from a decoded offset associated with the sameoffset type and the same offset class as those of the offset for which aprediction value will be determined in a case in which the offset typeassociated with the offset belongs to a second offset type group,thereby making it possible to improve the prediction precision while theamount of processing is reduced. Thus, with the above-describedconfiguration, it is possible to appropriately decode an offset fromcoded data having a smaller amount of data while the amount ofprocessing is reduced.

The first offset type is a type in which each pixel in a unit areaassociated with the first offset type is classified as one of aplurality of classes in accordance with, for example, the mode of anedge in the vicinity of the pixel. The second offset type is a type inwhich each pixel in a unit area associated with the second offset typeis classified as one of a plurality of classes in accordance with, forexample, the pixel value of the pixel.

The offset calculating means may preferably calculate each offset as alinear function of a prediction value determined by the prediction valuedetermining means and an offset residual decoded by the offset residualdecoding means. A coefficient to be multiplied by the prediction valuemay preferably differ depending on whether the offset type associatedwith the offset belongs to the first offset type group or the secondoffset type group.

With the above-described configuration, the coefficient to be multipliedby the prediction value differs depending on whether the offset typeassociated with the offset belongs to the first offset type group or thesecond offset type group. It is thus possible to calculate the offset byusing a more suitable coefficient depending on the offset type, therebymaking it possible to improve the coding efficiency.

The prediction value determining means may preferably determine aprediction value of each offset by calculating a weighted average of adecoded offset and a prediction value of the decoded offset.

With the above-described configuration, a prediction value of eachoffset is determined by calculating a weighted average of a decodedoffset and a prediction value of the decoded offset. Accordingly, aplurality of decoded offsets contribute to a prediction value of eachoffset, thereby suppressing excessive fluctuations in the predictionvalue. With this arrangement, for example, even if an inappropriateprediction value has been calculated due to the influence of noise, theinfluence of such an inappropriate prediction value can be suppressed,thereby making it possible to improve the coding efficiency.

The prediction value determining means may preferably include clippingmeans for clipping each of the determined prediction values by using anupper limit value and a lower limit value which correspond to a bitdepth of the pixel value of each pixel forming the input image.

With the above-described configuration, since each of the determinedprediction values is clipped by using an upper limit value and a lowerlimit value which correspond to a bit depth of the pixel value of eachpixel forming the input image, excessively large prediction values andexcessively small prediction values are not generated, thereby making itpossible to improve the coding efficiency.

An image filtering device according to the present invention is an imagefiltering device which operates on an input image. The image filteringdevice includes: calculating means for calculating a difference valuebetween a pixel value of a subject pixel forming an input image and apixel value of a pixel around the subject pixel; bit shift means forperforming bitwise right shift on a pixel value referred to by thecalculating means or the difference value calculated by the calculatingmeans by an amount equal to a predetermined shift value; classifyingmeans for classifying the subject pixel as one of a plurality of offsetclasses in accordance with a magnitude relation between the differencevalue subjected to bitwise right shift by the bit shift means and 0; andoffset means for adding an offset associated with the offset class ofthe subject pixel classified by the classifying means to the pixel valueof the subject pixel.

In the image filtering device configured as described above, the subjectpixel is classified as one of a plurality of offset classes inaccordance with a magnitude relation between the difference valuesubjected to bitwise right shift by the bit shift means and 0, and anoffset associated with the offset class of the subject pixel classifiedby the classifying means is added to the pixel value of the subjectpixel. Thus, the classifying processing is less vulnerable to theinfluence of noise, thereby making it possible to improve the codingefficiency.

The predetermined shift value may preferably have a positive correlationwith a bit depth of the pixel value of the subject pixel.

With the above-described configuration, since the predetermined shiftvalue may preferably have a positive correlation with a bit depth of thepixel value of the subject pixel, the coding efficiency can be moreeffectively improved.

An image filtering device according to the present invention is an imagefiltering device which operates on an input image. The image filteringdevice includes: calculating means for calculating a difference valuebetween a pixel value of a subject pixel forming an input image and apixel value of a pixel around the subject pixel; classifying means forclassifying the subject pixel as one of a plurality of offset classes inaccordance with a magnitude relation between the difference valuecalculated by the calculating means and each of predetermined first andsecond thresholds; and offset means for adding an offset associated withthe offset class of the subject pixel classified by the classifyingmeans to the pixel value of the subject pixel.

In the image filtering device configured as described above, the subjectpixel is classified as one of a plurality of offset classes inaccordance with a magnitude relation between the difference valuecalculated by the calculating means and each of the predetermined firstand second thresholds, and an offset associated with the offset class ofthe subject pixel classified by the classifying means is added to thepixel value of the subject pixel. Thus, the classifying processing isless vulnerable to the influence of noise, thereby making it possible toimprove the coding efficiency.

Absolute values of the first and second thresholds may preferably have apositive correlation with a bit depth of the pixel value of the subjectpixel.

With the above-described configuration, since the absolute values of thefirst and second thresholds have a positive correlation with the bitdepth of the pixel value of the subject pixel, the coding efficiency canbe more effectively improved.

An image filtering device according to the present invention is an imagefiltering device which operates on an input image constituted by aplurality of unit areas. The image filtering device includes:determining means for determining, among first and second offset types,an offset type to which a subject unit area including a subject pixelforming the input image belongs; classifying means for classifying thesubject pixel as one of an offset class in which an offset is not addedand a plurality of offset classes in which an offset is added inaccordance with the offset type to which the subject unit area belongsand a pixel value of the subject pixel; and offset means for adding anoffset associated with the offset type to which the subject unit areabelongs and the offset class of the subject pixel classified by theclassifying means to the pixel value of the subject pixel. In a case inwhich the pixel value of the subject pixel is within a predeterminedrange, the classifying means classifies the subject pixel as an offsetclass in which an offset is added, regardless of whether the offset typeto which the unit area including the subject pixel belongs is the firstoffset type or the second offset type.

In the image filtering device configured as described above, in a casein which the pixel value of the subject pixel is within a predeterminedrange, the subject pixel is classified as an offset class in which anoffset is added, regardless of whether the offset type to which the unitarea including the subject pixel belongs is the first offset type or thesecond offset type, thereby making it possible to effectively eliminateblock noise. Accordingly, with the above-described configuration, thecoding efficiency can be improved.

An image filtering device according to the present invention is an imagefiltering device for adding an offset to a pixel value of each pixelforming an input image which is constituted by a plurality of unitareas. The image filtering device includes: determining means fordetermining an offset type to which a subject unit area belongs among aplurality of offset types; offset coding means for determining an offsethaving a bit width which differs depending on the offset type and forcoding the offset; and filtering means for adding the determined offsetto the pixel value of each pixel forming the input image.

In the image filtering device configured as described above, among aplurality of offset types, the offset type to which a subject unit areabelongs is determined, an offset having a bit width which differsdepending on the determined offset type is determined, and thedetermined offset is added to the pixel value of each pixel forming theinput image. The determined offset is also coded.

Accordingly, with the above-described configuration, it is possible toperform appropriate offset filtering processing while the memory size ofa memory for storing offsets is reduced. With the above-describedconfiguration, since the amount of data required to code data isreduced, the coding efficiency is improved.

An offset coding device according to the present invention is an offsetcoding device for coding each offset which is referred to by an imagefilter for adding an offset to a pixel value of each pixel forming aninput image. The offset coding device includes: prediction valuedetermining means for determining a prediction value of each offset froma coded offset; offset residual calculating means for calculating anoffset residual from each offset and a prediction value determined bythe prediction value determining means; and offset residual coding meansfor coding an offset residual calculated by the offset residualcalculating means.

In the offset coding device configured as described above, there areprovided the prediction value determining means for determining aprediction value of each offset from a coded offset, the offset residualcalculating means for calculating an offset residual from each offsetand a prediction value determined by the prediction value determiningmeans, and the offset residual coding means for coding an offsetresidual calculated by the offset residual calculating means. It is thuspossible to reduce the amount of data required to code data can bereduced.

A data structure of coded data according to the present invention is adata structure of coded data which is referred to by an image filter foradding an offset to a pixel value of each pixel forming an input imagewhich is constituted by a plurality of unit areas. The data structureincludes: offset-type specifying information which specifies an offsettype to which each unit area belongs; and an offset having a bit widthwhich differs depending on the offset type. The image filter refers tothe offset-type specifying information included in the coded data, anddetermines an offset type to which a subject unit area belongs and alsodecodes an offset having a bit width which differs depending on thedetermined offset type.

The coded data configured as describe above includes an offset having abit width which differs depending on the offset type, thereby reducingthe amount of data required to code data. The image filter which decodesthe coded data refers to the offset-type specifying information, anddetermines the offset type to which a subject unit area belongs and alsodecodes an offset having a bit width which differs depending on thedetermined offset type. It is thus possible to perform appropriateoffset filtering processing while the memory size of a memory forstoring offsets is reduced.

The offset-type specifying information may be determined for each of theinput images or for each of the unit areas. Alternatively, theoffset-type specifying information may be determined for eachpredetermined set of the input images or for each predetermined set ofthe unit areas.

Third Embodiment

Offset information OI of a third embodiment will first be describedbelow with reference to FIG. 23. Part (a) of FIG. 23 illustrates syntax(indicated by “sao_offset_param( )” in part (a) of FIG. 23) of offsetinformation OI.

As shown in part (a) of FIG. 23, the offset information OI includes aparameter “sao_type_idx[sao_curr_depth][ys][xs]”. If the parameter“sao_type_idx[sao_curr_depth][ys][xs]” is not 0, a parametersao_offset[sao_curr_depth][ys][xs][i] is included in the offsetinformation OI.

(sao_curr_depth, ys, xs)

An argument “sao_curr_depth”, which is an argument of “sao_type_idx” and“sao_offset”, is a parameter indicating the split depth of a QAOU, and“ys” and “xs” are parameters respectively indicating the position in they direction and the position in the x direction of a QAOU (or a QAOMU,which will be discussed later).

The split modes of a QAOU in accordance with the values of“sao_curr_depth” are the same as those discussed with reference to FIG.4.

Part (b) of FIG. 23 illustrates syntax (indicated by “sao_split_param()” in part (b) of FIG. 23) of QAOU information. As indicated by thesyntax shown in part (b) of FIG. 23, if the split depth “sao_curr_depth”is smaller than a predetermined maximum value set by “saoMaxDepth”, itis determined by the parameter “sao_split_flag” whether a QAOU will befurther split. If the QAOU will be further split, a subsequenthierarchical depth “sao_split_param( )” is recursively called. If thesplit depth has reached the maximum value (“sao_curr_depth” is notsmaller than “saoMaxDepth”), “0” is set in“sao_split_flag[sao_curr_depth][ys][xs]”, and the QAOU will not befurther split.

FIG. 44 illustrates another example of syntax of offset information andsyntax of QAOU information.

Part (a) of FIG. 44 illustrates syntax of offset information. Theconfiguration of the syntax is similar to that shown in part (a) of FIG.23, but “component” indicating the value of a color component is addedas an argument of “sao_offset_param( )” and as an index of an array of“sao_split_flag”, “sao_type_idx”, and “sao_offset”. By the addition of“component”, a QAOU may be split differently according to a colorcomponent, such as a luminance component or a chrominance component, anddifferent offsets may be applied to split QAOUs.

Part (b) of FIG. 44 illustrates syntax of QAOU information. As in part(a) of FIG. 44, a color component “component” is added as an argument tothe syntax shown in part (b) of FIG. 23.

Part (c) of FIG. 44 illustrates the entire syntax of an adaptive offsetfilter which calls syntax of part (a) and syntax of part (b) of FIG. 44.The parameter “sao_flag” is a flag indicating whether or not an adaptiveoffset filter is applied. Only when this flag is true, will thefollowing parameters concerning the adaptive offset filter be used. Ifthis flag is true, the syntax “sao_split_param( )” and the syntax“sao_offset_param( )” are called by specifying the highest hierarchicallevel for each color component. Since the highest hierarchical level isspecified, arguments provided to each syntax are sao_curr_depth=0, ys=0,and xs=0. The color components are distinguished from each other bysetting the values of “component” as follows: the value is 0 in the caseof the luminance (Y), the value is 1 in the case of the chrominance(Cb), and the value is 2 in the case of the chrominance (Cr). Othervalues may be used as the values of the component as long as the colorcomponents can be distinguished from each other.

Concerning the chrominance (Cb) and the chrominance (Cr), flags“sao_flag_cb” and “sao_flag_cr” are respectively used to determinewhether an adaptive offset filter will be applied, and if it is notapplied, QAOU information and offset information concerning theassociated color component are not stored.

Since the argument “component” is added in the syntax shown in FIG. 44,in the description using FIG. 23, processing is performed by replacingthe argument [sao_curr_depth][ys][xs] by[sao_curr_depth][ys][xs][component]. In the subsequent description,processing is performed in a similar manner.

(Video Decoding Device 1′)

A video decoding device 1′ of the third embodiment will be describedbelow with reference to FIGS. 22 and 24 through 29. Elements having thesame functions as those discussed in the above-described embodiments aredesignated by like reference numerals, and an explanation thereof willthus be omitted.

As in the video decoding device 1, the video decoding device 1′partially includes a method defined in H. 264/MPEG-4. AVC, a method usedin KTA software, which is a joint development codec in VCEG (VideoCoding Expert Group), a method used in TMuC (Test Model underConsideration) software, which is a successor codec to the codec used inKTA software, and a technology used in HM (HEVC TestModel) software. Thevideo decoding device 1′ of this embodiment is different from the videodecoding device 1 of the first embodiment in that it includes anadaptive offset filter 60′ instead of the adaptive offset filter 60 ofthe video decoding device 1. The other elements of the configuration ofthe video decoding device 1′ are similar to those of the video decodingdevice 1.

(Adaptive Offset Filter 60)

Details of the adaptive offset filter 60′ will now be discussed belowwith reference to FIG. 22. FIG. 22 is a block diagram illustrating theconfiguration of the adaptive offset filter 60′. As shown in FIG. 22,the adaptive offset filter 60′ includes an adaptive offset filterinformation decoder 61′ and an adaptive offset filter processor 62′.

The adaptive offset filter information decoder 61′ includes, as shown inFIG. 22, an offset information decoding section 611 and a QAOU structuredecoding section 612.

The offset information decoding section 611 refers to QAOU informationincluded in the coded data #1 and decodes offset information OI includedin the QAOU information. The values“sao_type_idx[sao_curr_depth][ys][xs][component]” and“sao_offset[sao_curr_depth][ys][xs][i]” obtained by decoding the offsetinformation OI are supplied to the offset information storage section621 in association with the arguments (sao_curr_depth, ys, xs) and thearguments (sao_curr_depth, ys, xs, i).

More specifically, the offset information decoding section 611 decodescode from the coded data #1 and transforms the decoded code to the valueof “sao_type_idx”, and supplies it to the offset information storagesection 621 in association with the arguments. The offset informationdecoding section 611 changes the code decoding method and transform fromthe code into the value of “sao_type_idx” in accordance with thehierarchical depth of a subject QAOU to be processed. Conditions, suchas the hierarchical depth of a QAOU, are referred to as “parameterconditions”. Decoding of offset information in accordance with generalparameter conditions will be discussed later with reference to FIGS. 41and 42.

In the code decoding method, different maximum values may be useddepending on whether the hierarchical depth of a QAOU is smaller than athreshold or is equal to or greater than the threshold. Alternatively,the maximum value may be used only when the hierarchical depth of asubject QAOU is equal to or greater than the threshold. In the codedecoding method, different binarization techniques may be used dependingon whether the hierarchical depth of a subject QAOU is smaller than athreshold or is equal to or greater than the threshold. Alternatively,different contexts may be used depending on whether the hierarchicaldepth of a subject QAOU is smaller than a threshold or is equal to orgreater than the threshold.

For example, if the hierarchical depth of a subject QAOU is smaller thana threshold, code may be decoded by using variable-length coding(ue(v)), and if the hierarchical depth of a subject QAOU is equal to orgreater than the threshold, code may be decoded by using truncatedcoding (te(v)) corresponding to the number of offset types. If thenumber of offset types is a power of two, code may be decoded by usingfixed-length coding. If the number of offset types is four, it can berepresented by two bits, and thus, code may be decoded by usingfixed-length coding with two bits.

The offset information decoding section 611 transforms decoded code intothe value of “sao_type_idx” by using transform tables 801 and 802 shownin FIG. 25. The transform table 801 shown in part (a) of FIG. 25indicates two transform patterns. Among the two transform patterns, onepattern is used depending on the hierarchical depth of a subject QAOU.That is, when the hierarchical depth of a subject QAOU is smaller than athreshold, the offset information decoding section 611 utilizes atransform table 801A, and when the hierarchical depth of a subject QAOUis equal to or greater than the threshold, the offset informationdecoding section 611 utilizes a transform table 801B.

In the transform table 801A shown in part (a) of FIG. 25, an offset type“EO_0” (“sao_type_idx”=1) is associated with a code “1”, “EO_1”(“sao_type_idx”=2) is associated with a code “2”, and “EO_2”(“sao_type_idx”=3) is associated with a code “3”. Offset types areassociated with codes 4 through 6 in a similar manner.

In the transform table 801B shown in part (b) of FIG. 25, an offset type“EO_0” (“sao_type_idx”=1) is associated with a code “1”, “BO_0”(“sao_type_idx”=5) is associated with a code “2”, and “BO_1”(“sao_type_idx”=6) is associated with a code “3”. In the transform table801B, codes 4 through 6 are not used.

In this manner, in the transform table 801A, all the offset types usedfor the adaptive offsets (SAOs) are included. In contrast, in thetransform table 801B, only some of the offset types used for theadaptive offsets (SAOs) are included, and thus, when the hierarchicaldepth of a subject QAOU is equal to or greater than the threshold, someoffset types cannot be used.

The reason for this is as follows. Since the area of a QAOU becomessmaller in a deeper hierarchical level, the characteristics of pixelvalues within such a QAOU are almost uniform, and appropriate offsetscan be determined without using many offset types. Accordingly, thenumber of offset types can be decreased, thereby reducing a requiredmemory space. Additionally, the data length of coded data indicating theoffset types can be decreased, thereby making it possible to improve thecoding efficiency.

If the order of the associations between the codes and the offset typesin the transform table 801B is the same as that in the transform table801A, only the transform table 801A may be used.

A transform table 802 shown in part (b) of FIG. 25 indicates variousexamples of transform tables which are similar to the transform table801B and which may be used instead of the transform table 801B. In allthe transform tables, the mode of types is restricted, as in thetransform table 801B. The blank indicates that the associated code isnot used.

A transform table 802A is an example in which only edge offsets are usedand band offsets are not included. The number of offsets of an edgeoffset type is generally 4, and the number of offsets of a band offsettype is generally 16. In this manner, the number of offsets of an edgeoffset type is smaller than that of a band offset. Accordingly, byrestricting the use of band offsets, the memory space used for storingoffsets can be reduced. In particular, when the hierarchical depthbecomes deeper, the number of offsets is increased, and thus, the memoryspace is also increased. When the hierarchical depth becomes deeper, itis less likely that a band offset will be selected. Accordingly, thehierarchical depth is used as a parameter condition, and when thehierarchical depth is deep, the transform table 802A including only edgeoffsets is used, and when the hierarchical depth is not deep, atransform table (for example, the transform table 801A) including edgeoffsets and band offsets is used. It is thus possible to reduce thememory space without decreasing the coding efficiency. Additionally, thecost calculations for unnecessary options can be omitted in the codingdevice, thereby reducing the amount of processing.

A transform table 802B is an example in which only band offsets are usedand edge offsets are not included. A band offset requires a smalleramount of calculation than an edge offset. Additionally, a band offsetdoes not utilize a pixel around a subject pixel, and thus, a line memoryfor storing a reference pixel is not necessary. Accordingly, thetransform table 802B is used in accordance with the parameter condition,thereby achieving the above-described advantages. A transform table 802Cis an example in which only one band offset “BO_0” (“sao_type_idx”=5) isused.

A transform table 802D is an example in which one edge offset and twoband offsets are used, as in the transform table 801B. Morespecifically, the transform table 802D is an example in which offsettypes “EO_0” (“sao_type_idx”=1), “BO_0” (“sao_type_idx”=5), and “BO_1”(“sao_type_idx”=6) are used and shorter codes (smaller code numbers) arepreferentially assigned to the band offset types. When the hierarchicaldepth is not deep, it is more likely that a band offset will be selectedthan when the hierarchical depth is deep. Accordingly, the hierarchicaldepth is used as a parameter condition, and when the hierarchical depthis not deep, the transform table 802D is used, and a shorter code isassigned to the type which is more frequently used, thereby making itpossible to improve the coding efficiency.

A table other than those indicated by the examples shown in FIG. 25 maybe used, and the association between the codes and the offset types maybe changed in accordance with the parameter condition. An offset typewhich is different from any of the edge offsets and band offsets shownin the transform table 801A may be used singly or together with anotheroffset type in accordance with a condition, such as the hierarchicaldepth. Specific examples of such an offset type are an offset typehaving characteristics both of EO and BO, an edge offset which detectsan edge at a horizontal sample position different from that of a knownedge offset “EO_0”, or a band offset to which a band is allocated in amanner different from known band offsets “BO_0” and “BO_1”. Thesespecific examples will be discussed in another embodiment later.

The QAOU structure decoding section 612 decodes“sao_split_flag[sao_curr_depth][ys][xs]” included in the QAOUinformation so as to determine the QAOU split structure, and thensupplies QAOU structure information indicating the determined QAOU splitstructure to the offset information storage section 621.

An offset attribute setting section 613 which is discussed below may beincluded. The offset attribute setting section 613 determines the shiftvalue of an offset. An offset in coded data is coded with the offset bitdepth (also referred to as “SAO_DEPTH”) having a lower precision thanthe pixel bit depth (also referred to as “PIC_DEPTH”). That is, anoffset in the coded data is quantized. The shift value is a bit shiftamount which is necessary for performing inverse quantization. Theoffset attribute setting section 613 also determines the offset bitdepth and the offset value range. In this case, the offset bit depth isdetermined from the pixel bit depth (also referred to as “PIC_DEPTH”)(not shown) input into the offset attribute setting section 613. Thepixel bit depth represents the range of pixel values of pixels formingan image input into the adaptive offset filter 60′ as the bit width.When the pixel bit depth is N bits, the pixel values take a range of 0to 2^(N)−1.

The SAO bit depth and the shift value are determined by using thefollowing equations. However, other values may be used in accordancewith parameter conditions, which will be discussed later.SAO_DEPTH=MIN(PIC_DEPTH,10)shift value=PIC_DEPTH−MIN(PIC_DEPTH,10)

In one configuration, the offset value range (in this case, the maximumvalue) is determined by the following equation:Offset value range=2SAO_DEPTH−K−1 where K is a predetermined constant(discussed later).

The adaptive offset filter processor 62′ includes, as shown in FIG. 22,the offset information storage section 621, a QAOU controller 622, anoffset-type determining section 623, a classifying section 624, anoffset determining section 625, and an offset adder 626.

The offset information storage section 621 manages and stores the offsettype specified for each QAOU and specific values of offsets with respectto individual classes that can be selected for the specified offsettype, on the basis of the QAOU structure information,“sao_type_idx[sao_curr_depth][ys][xs]”, and“sao_offset[sao_curr_depth][ys][xs][i]”. The offset information storagesection 621 has a map memory and a list memory.

The map memory and the list memory will be discussed below withreference to FIG. 24. FIG. 24 illustrates examples of information storedin the map memory and the list memory. Part (a) of FIG. 24 illustratesexamples of QAOU indexes stored in the map memory, and part (b) of FIG.24 illustrates examples of information stored in the list memory.

In a map memory 601, a QAOU index assigned to each offset minimum unit(also referred to as “QAOMU: Quad Adaptive Offset Minimum Unit”), whichis determined by the split depth, is stored. The QAOU index will bediscussed later. Part (a) of FIG. 24 shows QAOMUs having a split depthof three and forming the unit of processing (for example, an LCU) andQAOU indexes assigned to QAOMUs. In part (a) of FIG. 24, indexes 0 to 9are assigned to QAOUs in a simple manner without considering the QAOUsplit depth. In the example shown in part (a) of FIG. 24, a QAOUspecified by QAOU index=I is indicated by QAOUI. In part (a) of FIG. 24,the thin lines indicate QAOMU boundaries, while the thick lines indicateQAOU boundaries.

As shown in part (a) of FIG. 24, QAOU0 is constituted by four QAOMUs,and 0 is assigned to these four QAOMUs as the QAOU index. QAOU3 isconstituted by one QAOMU, and 3 is assigned to this QAOMU as the QAOUindex. In this manner, in the map memory, QAOU indexes assigned to theindividual QAOMUs are stored.

In a list memory 602, the offset type associated with each QAOU indexand specific values of offsets with respect to classes that can beselected for this offset type are stored in association with the QAOUindex.

This will be more specifically discussed below with reference to part(b) of FIG. 24. Part (b) of FIG. 24 shows an offset type associated witheach of the QAOU indexes 0 to 9 and offsets with respect to individualclasses that can be selected for each offset type. In part (b) of FIG.24, “xxx” indicates specific numeric values that can be different fromeach other.

In part (b) of FIG. 24, “BO_1” indicates an offset type specified by“sao_type_idx”=5. “EO_1” indicates an offset type specified by“sao_type_idx”=1. In this manner, edge offsets, which are of an offsettype specified by “sao_type_idx”=1, 2, 3, 4, are also indicated by EO_1,2, 3, 4, respectively, and band offsets, which are of an offset typespecified by “sao_type_idx”=5, 6, are also indicated by BO_1, 2,respectively.

As shown in part (b) of FIG. 24, when the offset type is a band offset,a total of sixteen offsets, that is, the offset 1 through the offset 16,are stored in the list memory for this offset type. The offset 1 throughthe offset 16 are values specified by“sao_offset[sao_curr_depth][ys][xs][1]” through“sao_offset[sao_curr_depth][ys][xs][16]”, respectively, when the valueof “sao_type_idx[sao_curr_depth][ys][xs]” is “5” or “6”.

On the other hand, when the offset type is an edge offset, a total offour offsets, that is, the offset 1 through the offset 4, are stored inthe list memory for this offset type. The offset 1 through the offset 4are values specified by “sao_offset[sao_curr_depth][ys][xs][1]” through“sao_offset[sao_curr_depth][ys][xs][4]”, respectively, when the value of“sao_type_idx[sao_curr_depth][ys][xs]” is one of “1, 2, 3, and 4”. Inthe case of an edge offset, no value is stored in the offset 5 throughthe offset 16.

A QAOMU number is appended to each QAOMU, and the QAOMUs can bedistinguished from each other by the QAOMU numbers. Hereinafter, QAOMUhaving a QAOMU number NQ will also be referred to as “QAOMUN_(Q)”.

In this embodiment, the offset precision may be varied in accordancewith the hierarchical depth of a subject QAOU. The shift value used forinverse quantization also differs depending on the offset precision. Inthis case, too, the offset precision and the shift value are determinedby using the pixel bit depth PIC_DEPTH.

As an example of the offset precision, the offset precision may bevaried, for example, as follows. When the hierarchical depth of a QAOUis smaller than a threshold, the offset precision and shift value areset:SAO_DEPTH=MIN(PIC_DEPTH,10)

shift value=PIC_DEPTH−MIN(PIC_DEPTH, 10); and when the hierarchicaldepth of a QAOU is equal to or greater than the threshold, the offsetprecision and shift value are set:SAO_DEPTH=MIN(PIC_DEPTH,8)shift value=PIC_DEPTH−MIN(PIC_DEPTH,8).

As stated above, the offset precision (offset bit depth, SAO_DEPTH) andthe pixel bit depth (PIC_DEPTH), which expresses the range of pixelvalues of pixels forming an input image as the bit width, are closelyrelated to each other in terms of quantization errors. The bit depth ofan image output from the adaptive offset filter 60 is represented by thepixel bit depth PIC_DEPTH, and SAO_DEPTH represents the bit depth of anoffset to be added to a pixel. Accordingly, even if an offset having ahigher precision than that of the pixel bit depth is used, it isdiscarded in the output process. It is thus meaningless to set SAO_DEPTHwhich exceeds PIC_DEPTH. On the other hand, if SAO_DEPTH is lower thanPIC_DEPTH, an input image is corrected merely with a precision lowerthan the precision (PIC_DEPTH) with which the input image could becorrected by using a filter, thereby decreasing the filtering effect.

Accordingly, in this embodiment, by restricting a quantized offset to anoffset value range that can be represented by a certain bit width, it ispossible to reduce the bit width for storing a quantized offset in theoffset information storage section 621. With this arrangement, theeffect of reducing the memory size can be obtained, compared with theabove-described restriction is not imposed. However, if the offset valuerange is excessively restricted, the effect of correcting distortion ofa decoded image by using an offset is decreased. Accordingly, it is notpossible to remove distortion of a decoded image even by offset additionprocessing, which may decrease the coding efficiency.

Thus, by setting the offset value range in an optimal range in such adegree as not to decrease the coding efficiency, the filtering effectcan be maintained while a memory space to be used is reduced.

The maximum bit length representing the offset value range is indicatedby CLIP_BIT, and by calculating CLIP_BIT=SAO_DEPTH−K, the offset valuerange is determined to be −2^(CLIP_BIT−1) to 2^(CLIP_BIT−1)−1. Then, ithas been found through the inventors' experiments that in the case ofK=4, the coding efficiency was not decreased even if the offset rangewas restricted by the offset value range.

When the pixel bit depth is eight, the offset bit depth SAO_DEPTH isalso eight, and thus, CLIP_BIT=8-K=4. If one offset can be stored byusing four bits, in software, for example, which handles one byteconstituted by eight bits as the unit, one offset can be packed andstored in one byte, thereby easily reducing the memory size.

If the offset precision to be set differs depending on the hierarchicaldepth of a subject QAOU, the offset information storage section 621secures an offset storage area by changing the unit size of the offsetstorage area in accordance with the hierarchical depth of the subjectQAOU.

More specifically, it is now assumed that when the hierarchical depth ofa subject QAOU is smaller than a threshold, the offset precision is setto be n_(A) bits (for example, n_(A)=8 or 6), and that when thehierarchical depth of the subject QAOU is equal to or greater than thethreshold, the offset precision is set to be n_(B) bits (n_(A)>n_(B),for example, n_(B)=n_(A)−2). In this case, the offset informationstorage section 621 secures an area for storing offsets when thehierarchical depth of the subject QAOU is smaller than the threshold byusing the n_(A) bits as the unit, and secures an area for storingoffsets when the hierarchical depth of the subject QAOU is equal to orgreater than the threshold by using the n_(B) bits as the unit.

If the hierarchical depth of the subject QAOU is equal to or greaterthan the threshold, when reading and writing an offset from and into theoffset information storage section 621, it is preferable that an offsetis written by rounding it down by (n_(A)−n_(B)) bits and that an offsetis read by rounding it up by (n_(A)−n_(B)) bits. With this input/outputconfiguration, it is not necessary for other modules to performprocessing by considering the difference in the offset precision.

If the required number of classes differs depending on the hierarchicaldepth of a subject QAOU, the offset information storage section 621changes an area of a list memory to be secured in accordance with therequired number of classes. For example, a band offset which is splitinto sixteen classes if the hierarchical depth of a subject QAOU issmaller than a threshold and which is split into eight classes if thehierarchical depth of the subject QAOU is equal to or greater than thethreshold, will be considered (such a band offset will be discussedlater). In this case, the memory space necessary for securing classeswhen the hierarchical depth of the subject QAOU is equal to or greaterthan the threshold is half the memory space when the hierarchical depthof the subject QAOU is smaller than the threshold. Accordingly, when thehierarchical depth of the subject QAOU is equal to or greater than thethreshold, the offset information storage section 621 changes the sizeof the list memory to be secured to be half the memory size to be setwhen the hierarchical depth of the subject QAOU is smaller than thethreshold.

The QAOU controller 622 controls the individual components included inthe adaptive offset filter processor 62′. The QAOU controller 622 alsorefers to the QAOU structure information, and then splits the deblockeddecoded image P_DB into one or a plurality of QAOUs and scans theindividual QAOUs in a predetermined order. The QAOU controller 622 alsosupplies a QAOMU number representing a subject QAOMU to the offset-typedetermining section 623.

The offset-type determining section 623 refers to the map memory and thelist memory of the offset information storage section 621 and determinesthe offset type specified by the QAOMU number supplied from the QAOUcontroller 622. The offset-type determining section 623 also suppliesthe determined offset type to the classifying section 624.

The classifying section 624 classifies each pixel included in thesubject QAOU as one of the classes that can be selected for the offsettype supplied from the offset-type determining section 623. Theclassifying section 624 also supplies the offset type and the classindex indicating the class of each pixel to the offset determiningsection 625. Specific classifying processing performed by theclassifying section 624 will be discussed later, and thus, anexplanation thereof is omitted here.

The offset determining section 625 refers to the list memory of theoffset information storage section 621, and determines, for each pixelincluded in the subject QAOU, an offset specified by the offset type andthe class index supplied from the classifying section 624. The offsetdetermining section 625 includes an offset reverse shifter (not shown)which performs bitwise left shift on an offset by an amount equal to theshift value set by the offset attribute setting section 613. The offsetreverse shifter performs inverse quantization on the offset so that theoffset bit depth may match the pixel bit depth. By performing suchinverse quantization, in addition processing performed by the offsetadder 626, which will be discussed later, the offset can be added to thepixel value by using the same bit depth. The inverse-quantized offsetfor each pixel is supplied to the offset adder 626.

The offset adder 626 adds an offset supplied from the offset determiningsection 625 to each pixel which forms the subject QAOU of the deblockeddecoded image P_DB. The offset adder 626 outputs an image obtained byperforming processing on all the QAOUs included in the deblocked decodedimage P_DB as an offset-filtered decoded image P_OF.

A description will now be given of classifying processing performed bythe classifying section 624 with reference to FIGS. 26 through 29.Classifying processing performed by using a condition, such as thehierarchical depth of a QAOU, as a parameter condition, will bediscussed below. The configuration of a classifying section based ongeneral parameter conditions will be discussed later with reference toFIG. 34.

FIG. 26 illustrates offset processing performed by the adaptive offsetfilter 60. Part (a) of FIG. 26 shows graphs indicating the magnituderelations between the pixel value pic[x] of a subject pixel x and thepixel value of a pixel a or b and also shows the values of a functionSign in accordance with the magnitude relation. Part (b) of FIG. 26shows graphs indicating the magnitude relations between the pixel valueof the subject pixel x and each of the pixel values of the pixel a andthe pixel b and also shows the value of EdgeType in accordance with themagnitude relation. Part (c) of FIG. 26 shows the association betweeneach graph shown in part (b) of FIG. 26 and class_idx. Parts (d) through(f) of FIG. 26 illustrate transform tables used for transforming fromEdgeType to class_idx.

FIG. 27 illustrates offset processing performed by the adaptive offsetfilter 60: part (a) schematically shows classifying performed when“sao_type_idx=” 5; and part (b) schematically shows classifyingperformed when “sao_type_idx6”.

FIG. 28 illustrates offset processing performed by the adaptive offsetfilter 60: part (a) schematically shows classifying performed when thehierarchical depth of a subject QAOU is smaller than a threshold; andpart (b) schematically shows classifying performed when the hierarchicaldepth of the subject QAOU is equal to or greater than the threshold.

FIG. 29 illustrate an example of classifying processing when a bandoffset is specified: part (a) shows an example of classifying processingwhen the hierarchical depth of a subject QAOU is smaller than athreshold; and part (b) shows an example of classifying processing whenthe hierarchical depth of the subject QAOU is equal to or greater thanthe threshold.

(When Offset Type is One of 1 to 4 (Edge Offset))

When the offset type supplied from the offset-type determining section623 is one of 1 to 4, the classifying section 624 performs the sameprocessing as that discussed with reference to FIG. 10.

Accordingly, as shown in part (a) of FIG. 26, concerning the magnituderelation between the pixel value pic[x] and the pixel value of the pixela or b, when the pixel value pic[x] is smaller than the pixel value ofthe pixel a or b, Sign(pic[x]−pix[a])=−1, when the pixel value pic[x] isthe same as the pixel value of the pixel a or b, Sign(pic[x]−pix[a])=0,and when the pixel value pic[x] is greater than the pixel value of thepixel a or b, Sign(pic[x]−pix[a])=1. In the graphs shown in part (a) ofFIG. 26, the black circle with pic[x] indicates the pixel value of thesubject pixel x, and the black circle without pic[x] indicates the pixelvalue of the subject pixel a or b. The vertical direction in the graphsshown in part (a) of FIG. 26 indicates the magnitude of the pixel value.

Then, the classifying section 624 finds EdgeType according to thefollowing mathematical equation (1-1) on the basis ofSign(pic[x]−pix[a]) and Sign(pic[x]−pix[b]).EdgeType=Sign(pic[x]−pix[a])+Sign(pic[x]−pix[b])+2  (1-1)According to this equation, as shown in part (b) of FIG. 26, when thepixel values of both of the pixels a and b are greater than the pixelvalue pic[x], EdgeType=0. When the pixel value of one of the pixels aand b is greater than the pixel value pic[x] and when the pixel value ofthe other pixel a or b is the same as the pixel value pic[x],EdgeType=1. When the pixel value of one of the pixels a and b is smallerthan the pixel value pic[x] and when the pixel value of the other pixela or b is the same as the pixel value pic[x], EdgeType=3. When the pixelvalues of both of the pixels a and b are smaller than the pixel valuepic[x], EdgeType=4. When the pixel value of one of the pixels a and b issmaller than the pixel value pic[x] and when the pixel value of theother pixel a or b is greater than the pixel value pic[x], or when thepixel values of both of the pixels a and b are the same as the pixelvalue pic[x], EdgeType=2.

In part (b) of FIG. 26, the center black circle in each graph indicatesthe pixel value of the subject pixel x, and the black circles on bothsides indicate the pixel values of the pixel a and the pixel b. Thevertical direction in the graphs shown in part (b) of FIG. 26 indicatesthe magnitude of the pixel value.

Then, on the basis of the determined EdgeType, the classifying section624 determines the class index (class_idx) of the class to which thesubject pixel x belongs in the following manner.class_idx=EoTbl[EdgeType]where EoTbl[EdgeType] is a transform table used for determiningclass_idx from EdgeType. Specific examples of the transform table EoTblare shown in parts (d) through (f) of FIG. 26.

A transform table 1001X shown in part (d) of FIG. 26 is a transformtable which is not switched in accordance with the hierarchical depth ofa subject QAOU.

A transform table 1001A shown in part (e) of FIG. 26 and a transformtable 1001B shown in part (f) of FIG. 26 are transform tables which areswitched in accordance with the hierarchical depth of a subject QAOU. Ifthe hierarchical depth of a subject QAOU is smaller than a threshold,the transform table 1001A is used, and if the hierarchical depth of thesubject QAOU is equal to or greater than the threshold, the transformtable 1001B is used.

When the same transform table is used regardless of the hierarchicaldepth of a subject QAOU, as indicated by the transform table 1001X, ifthere is no edge in an area constituted by the subject pixel x, thepixel a, and the pixel b (hereinafter, such a case will also be referredto as a case in which an area is flat), that is, when EdgeType=2, theclassifying section 624 classifies the subject pixel x as class 0(“class_idx”=0). The classifying section 624 also classifies EdgeType=0,1, 3, 4 as class_idx=1, 2, 3, 4, respectively.

When the transform table is not switched in accordance with thehierarchical depth of a subject QAOU and when the hierarchical depth ofa subject QAOU is smaller than a threshold, as indicated by thetransform table 1001A, if there is no edge in an area constituted by thesubject pixel x, the pixel a, and the pixel b (hereinafter, such a casewill also be referred to as a case in which an area is flat), that is,when EdgeType=2, the classifying section 624 classifies the subjectpixel x as class 0 (“class_idx”=0). The classifying section 624 alsoclassifies EdgeType=0, 1, 3, 4 as class_idx=1, 3, 4, 2, respectively.When the hierarchical depth of the subject QAOU is equal to or greaterthan the threshold, as indicated by the transform table 1001B, if thereis no edge in an area constituted by the subject pixel x, the pixel a,and the pixel b (hereinafter, such a case will also be referred to as acase in which an area is flat), that is, when EdgeType=2, theclassifying section 624 classifies the subject pixel x as class 0(“class_idx”=0). The classifying section 624 also classifies EdgeType=0,1 as class_idx=1 and EdgeType=3, 4 as class_idx=2. Accordingly, in thetransform table 1001B, one class index (class_id) is associated with aplurality of edge types (EdgeType).

It appears that the transform table 1001A may be formed in the samemanner as the transform table 1001X. However, in a case in which twotransform tables are switched in accordance with the hierarchical depth,if, as the transform table 1001A, the same table as that defined as thetransform table 1001X is used, different edge types (EdgeType) areapplied to the same class index (class_idx) in relation to the transformtable 1001B, and processing is different depending on the hierarchicaldepth. For example, in the case of class_idx=2, when the hierarchicaldepth is smaller than the threshold (when the transform table 1001X isused), the edge type is 1 (EdgeType=1), and when the hierarchical depthis equal to or greater than the threshold (when the transform table1001B is used), the edge type is 4 (EdgeType=4). Thus, processing isdifferent depending on the hierarchical depth.

Accordingly, by setting the transform table 1001A to be the transformtable 1001B shown in part (e) of FIG. 26, the same processing isperformed regardless of the hierarchical depth.

(When Offset Type is 5 or 6 (Band Offset))

When the offset type supplied from the offset-type determining section623 is 5 or 6, the classifying section 624 classifies the pixel value ofthe subject pixel x as one of a plurality of classes in accordance withthe pixel value pic[x] of the subject pixel x.

-   -   When offset type is 5 (sao_type_idx=5)

When the pixel value pic[x] of the subject pixel x satisfies:(max×¼)≤pic[x]≤(max×¾), the classifying section 624 classifies thesubject pixel as a class other than 0. That is, when the pixel value ofthe subject pixel is contained within a range indicated by the hatchedportion in part (a) of FIG. 11, the classifying section 624 classifiesthe subject pixel as a class other than 0. In the above-describedcondition, max indicates the maximum value that can be taken by thesubject pixel x, and for example, max=255. When max=255, theabove-described condition may be represented by 8≤(pic[x]/8)≤23, or4≤(pic[x]/16)≤11.

-   -   When offset type is 6 (sao_type_idx=6)

When the pixel value pic[x] of the subject pixel x satisfies:pic[x]≤(max×¼) or (max×¾)≤pic[x], the classifying section 624 classifiesthe subject pixel as a class other than 0. That is, when the pixel valueof the subject pixel is contained within a range indicated by thehatched portion in part (b) of FIG. 11, the classifying section 624classifies the subject pixel as a class other than 0. In theabove-described condition, max indicates the maximum value that can betaken by the subject pixel x, and for example, max=255. When max=255,the above-described condition may be represented by (pic[x]/8)≤7 or24≤(pic[x]/8), or (pic[x]/16)≤3 or 12≤(pic[x]/16).

The classifying processing performed by the classifying section 624 willbe more specifically described below.

When the offset type is 5 or 6, the classifying section 624 determinesthe class index (class_idx) of the class to which the subject pixel xbelongs in accordance with the hierarchical depth of a subject QAOU inthe following manner.

In a case in which the hierarchical depth of the subject QAOU is smallerthan a threshold,class_idx=EoTbl[sao_type_idx][pic[x]/>>BoRefBit32]; and

-   -   In a case in which the hierarchical depth of the subject QAOU is        equal to or greater than the threshold        class_idx=EoTbl[sao_type_idx][pic[x]/>>BoRefBit16]        where BoTbl[sao_type_idx][pic[x]/>>BoRefBit32] and        BoTbl[sao_type_idx][pic[x]/>>BoRefBit16] are transform tables        used for determining class_idx from the pixel value pic[x] of        the subject pixel x and sao_type_idx. When the image bit depth        is indicated by PIC_DEPTH, BoRefBit32 and BoRefBit16 are values        determined by PIC_DEPTH−5 and PIC_DEPTH−4, respectively, and are        values obtained by quantizing a pixel value in 32 or 16 steps.        The quantized pixel value is also indicated by pixquant.        Performing bitwise right shift by BoRefBit32 and BoRefBit16        corresponds to performing division by 1<<BoRefBit32 and        1<<BoRefBit16, respectively. The quantization widths of        1<<BoRefBit32 and 1<<BoRefBit16 are 1<<(8-5)=8 and 1<<(8-4)=16,        respectively, when the bit depth of the pixel values is eight        bits. Classifying processing when the quantization width is        eight will be discussed below. The quantization width is        referred to as a “class width”.

The classifying section 624 performs classifying by changing the classwidth in accordance with the hierarchical depth of a subject QAOU. Forexample, as shown in part (a) of FIG. 28, when the hierarchical depth ofa subject QAOU is smaller than a threshold, the classifying section 624divides the pixel values into 32 levels by setting the class width to be“8” and then performs classifying. As shown in part (b) of FIG. 28, whenthe hierarchical depth of a subject QAOU is equal to or greater than thethreshold, the classifying section 624 divides the pixel values into 16levels by setting the class width to be “16” and then performsclassifying.

Specific examples of the transform table EoTbl are shown in FIG. 29. Inparts (a) and (b) of FIG. 29, “BO_1” indicates that “sao_type_index”=5,and “BO_2” indicates that “sao_type_index”=6. A transform table 1301shown in part (a) of FIG. 29 is a transform table when the hierarchicaldepth of a subject QAOU is smaller than a threshold, and a transformtable 1302 shown in part (b) of FIG. 29 is a transform table when thehierarchical depth of a subject QAOU is equal to or greater than thethreshold.

When the hierarchical depth of a subject QAOU is smaller than thethreshold and when “sao_type_index”=5, as shown in part (a) of FIG. 29,the classifying section 624 classifies the subject pixel x as one ofclass indexes 1 to 16 in accordance with the magnitude of pic[x] if thepixel value pic[x] of the subject pixel x satisfies the condition8≤(pic[x]/8)≤23.

When “sao_type_idx”=6, the classifying section 624 classifies thesubject pixel x as one of class indexes 1 to 16 in accordance with themagnitude of pic[x] if the pixel value pic[x] of the subject pixel xsatisfies the condition (pic[x]/8)≤7 or 24≤(pic[x]/8).

When the hierarchical depth of a subject QAOU is equal to or greaterthan the threshold and when “sao_type_index”=5, as shown in part (b) ofFIG. 29, the classifying section 624 classifies the subject pixel x asone of class indexes 1 to 8 in accordance with the magnitude of pic[x]if the pixel value pic[x] of the subject pixel x satisfies the condition4≤(pic[x]/16)≤11.

When “sao_type_idx”=6, the classifying section 624 classifies thesubject pixel x as one of class indexes 1 to 8 in accordance with themagnitude of pic[x] if the pixel value pic[x] of the subject pixel xsatisfies the condition (pic[x]/16)≤3 or 12≤(pic[x]/16).

(Configuration of Offset Information Decoding Section 611)

FIG. 41 is a block diagram of the offset information decoding section611 which changes the offset type to be used in accordance with aparameter type (parameter condition) of an adaptive offset and/or whichchanges the number of classes in accordance with a parameter conditionof an adaptive offset. The offset information decoding section 611includes an adaptive offset type decoder 6111, an offset type selector6112, an offset type decoder 6113, an adaptive offset decoder 6114, anumber-of-offsets selector 6115, and an offset decoder 6116. Theparameter conditions are parameters other than values calculated frompixel values. Examples of the parameter conditions are a block size(QAOU size), a color component (component), and QP, which will bediscussed in appendixes later, in addition to the hierarchical depth andthe offset type described above.

The adaptive offset type decoder 6111 is means for adaptively decodingoffset types from QAOU information included in coded data in accordancewith a parameter condition, and includes the offset type selector 6112and the offset type decoder 6113.

The offset type selector 6112 is means for selecting the offset types tobe used in accordance with the parameter condition. One of the parameterconditions is the above-described hierarchical depth. The number ofoffset types to be used when the hierarchical depth is not deep issmaller than that when the hierarchical depth is deep. The offset typeselector 6112 inputs a transform table, such as that discussed withreference to FIG. 25, to the offset type decoder 6113. The offset typeselector 6112 also inputs the maximum number of usable offset types tothe offset type decoder 6113.

The offset type decoder 6113 decodes offset types from the inputtransform table and the maximum number. If the maximum number of offsettypes is N, the range of codes that can be taken is restricted to N from0 to N−1, thereby reducing the number of bits required for coding codes.For example, when the maximum value is greater than 2^(m-1) but notgreater than 2^(m), m-bit fixed-length coding can be used. Truncatedunary coding or Truncated Rice coding having N−1 as the maximum valuemay be used.

The adaptive offset decoder 6114 is means for adaptively decodingoffsets from QAOU information included in the coded data in accordancewith a parameter condition, and includes the number-of-offsets selector6115 and the offset decoder 6116. The number-of-offsets selector 6115 ismeans for selecting the maximum number of offsets to be used and theoffset precision in accordance with a parameter condition. One of theparameter conditions is the hierarchical depth. The number of offsetswhen the hierarchical depth is not deep is greater than that when thehierarchical depth is deep. For example, when the hierarchical depth isnot deep, sixteen offsets can be used, as shown in part (a) of FIG. 29,and when the hierarchical depth is deep, eight offsets can be used, asshown in part (b) of FIG. 29. Additionally, as stated above, the offsetprecision (bit depth) can be changed in accordance with the hierarchicaldepth. The number-of-offsets selector 6115 inputs the maximum number ofoffsets and the offset precision to the offset decoder 6116. The offsetdecoder 6116 decodes offsets in accordance with the maximum number ofoffsets and the offset precision. If the number of offsets is decreased,the amount of data to code offsets is reduced. Additionally, as in theoffset type, if the precision of each offset is determined, the range ofa code for coding each offset that can be taken is also restricted,thereby reducing the number of bits required for coding a code.

Part (a) of FIG. 42 is a block diagram of a configuration of the offsettype selector 6112.

The offset type selector 6112 includes an offset-type transform tableselector 6117, a first offset-type transform table storage section 6118,and a second offset-type transform table storage section 6119.

The offset-type transform table selector 6117 selects a transform tablestored in the first offset-type transform table storage section 6118 ora transform table stored in the second offset-type transform tablestorage section 6119 in accordance with a parameter condition. In theabove-described example, the transform table 801A is the transform tablestored in the first offset-type transform table storage section 6118,and the transform table 801B is the transform table stored in the secondoffset-type transform table storage section 6119.

Part (b) of FIG. 42 is a block diagram of another configuration of theoffset type selector.

An offset type selector 6112′ includes an offset type transform tableselector 6117′, an edge-offset-type-and-band-offset-type transform tablestorage section 6118′, and ahorizontal-edge-offset-type-and-band-offset-type transform table storagesection 6119′.

(Configuration of Classifying Section 624)

FIG. 43 is a block diagram of the configuration of the classifyingsection 624.

The classifying section 624 includes an adaptive edge-offset classifyingsection 6241, an edge-offset class selector 6242, an edge-offsetclassifying section 6243, an adaptive band-offset classifying section6244, a band-offset-class-and-class-width selector 6245, and aband-offset classifying section 6246.

The classifying section 624 classifies each pixel as a class inaccordance with a parameter condition and an offset type. When theoffset type indicates an edge offset, the classifying section 624classifies each pixel by using the adaptive edge-offset classifyingsection 6241. When the offset type indicates a band offset, theclassifying section 624 classifies each pixel by using the adaptiveband-offset classifying section 6244.

The adaptive edge-offset classifying section 6241 is means foradaptively classifying each pixel as a class in accordance with aparameter condition, and includes the edge-offset class selector 6242and the edge-offset classifying section 6243. The edge-offset classselector 6242 selects the type of class to be used. The edge-offsetclass selector 6242 inputs a classifying method for classifying pixelvalues into the edge-offset classifying section 6243. More specifically,the edge-offset class selector 6242 inputs a method for determining anintermediate value EdgeType which is temporarily determined when a pixelvalue is classified and a transform table used for determining class_idxfrom EdgeType. An example of the method for determining the intermediatevalue EdgeType has been discussed with reference to parts (a) and (b) ofFIG. 26, and this is referred to as a “basic edge classifying method”.Alternatively, an edge classifying method, which will be discussed laterwith reference to FIG. 33, may be used. One of the parameter conditionsis the hierarchical depth, as stated above, and the number of classes tobe used when the hierarchical depth is not deep is set to be greaterthan that when the hierarchical depth is deep. Examples of the transformtable used for determining class_idx from EdgeType are the transformtables 1001A and 1001B. A method for switching the edge determiningmethod in accordance with the color component (component) may besuitably used. In this case, for the luminance components, the basicedge classifying method for classifying edges in the horizontal,vertical, and oblique directions are suitably used, and, for thechrominance components, a horizontal edge classifying method is suitablyused in order to reduce the number of line memories. In the horizontaledge classifying method, the range of reference pixels used forclassifying edges is restricted to the horizontal direction with respectto a subject pixel. The edge-offset classifying section 6243 classifiespixels on the basis of the supplied classifying method and the transformtable used for determining class_idx from EdgeType.

The adaptive band-offset classifying section 6244 is means foradaptively classifying each pixel as a class in accordance with aparameter condition, and includes the band-offset-class-and-class-widthselector 6245 and the band-offset classifying section 6246. Theband-offset-class-and-class-width selector 6245 inputs a classifyingmethod for classifying pixel values into the band-offset classifyingsection 6246. More specifically, the band-offset-class-and-class-widthselector 6245 inputs a class width, which is a quantization width usedfor classifying a pixel value as an intermediate value, and a transformtable used for determining class_idx from the intermediate value. One ofthe parameter conditions is the hierarchical depth, as stated above, andthe class width to be used when the hierarchical depth is not deep issmaller than that when the hierarchical depth is deep. The band-offsetclassifying section 6246 classifies a pixel value as a class inaccordance with the received class width and the transform table usedfor determining class_idx from the intermediate value. The class width,which is an input, does not have to be a class width itself, and may bean integer corresponding to the class width. For example, if1<<BoRefBit32 is used as the class width, the number of bits BoRefBit32,which is the logarithm to the base 2, used for quantizing a pixel may beused instead of the class width, or the value for determining the numberof bits used for quantizing a pixel from the pixel bit depth, forexample, if BoRefBit32=PIC_DEPTH−5, 5 may be used instead of the classwidth.

In this manner, coded data is adaptively decoded in accordance with aparameter condition, and classifying of pixels is performed.

As described above, in this embodiment, the number of classes used forclassifying pixels is changed in accordance with the hierarchical depthof a subject QAOU. More specifically, when the hierarchical depth of asubject QAOU is deep, the number of classes is smaller than that whenthe hierarchical depth of the subject QAOU is not deep.

Generally, the required memory space is represented by: the memoryspace=the data length of an offset×the number of classes×the number ofQAOUs. Accordingly, by decreasing the number of classes, the memoryspace to be used can also be reduced.

In a deeper hierarchical level, the area of a QAOU becomes smaller, andalso, the characteristics of pixel values within such a QAOU are almostuniform. Accordingly, even if the number of classes is decreased, theeffect of offsets is not considerably impaired.

Additionally, in this embodiment, the offset precision is changed inaccordance with the hierarchical depth of a subject QAOU. Morespecifically, when the hierarchical depth of a subject QAOU is deep, theoffset precision is lower than that when the hierarchical depth of asubject QAOU is not deep. With this arrangement, the coding amountrequired when the hierarchical depth of a subject QAOU is deep can bereduced.

In a deeper hierarchical level of a QAOU, the number of pixels includedin such a hierarchical level is smaller, and thus, quantization errorsin the entire QAOU are also smaller than those in a higher hierarchicallevel. Accordingly, even if the offset precision is decreased in adeeper hierarchical level, the effect of offsets is not considerablyimpaired.

(Video Coding Device 2′)

A video coding device 2′ for generating coded data #1 by coding asubject image will be described below with reference to FIGS. 30 and 31.As in the video coding device 2, the video coding device 2′ partiallyincludes a method defined in H. 264/MPEG-4. AVC, a method used in KTAsoftware, which is a joint development codec in VCEG (Video CodingExpert Group), a method used in TMuC (Test Model under Consideration)software, which is a successor codec to the codec used in KTA software,and a technology used in HM (HEVC TestModel) software.

The video coding device 2′ includes an adaptive offset filter 80′instead of the adaptive offset filter 80 of the video coding device 2.The other elements of the configuration of the video coding device 2′are similar to those of the video coding device 2.

(Adaptive Offset Filter 80′)

An adaptive offset filter 80′ will be discussed below with reference toFIG. 30. FIG. 30 is a block diagram illustrating the configuration ofthe adaptive offset filter 80′. The adaptive offset filter 80′ includes,as shown in FIG. 30, an adaptive offset filter information setting unit81′ and an adaptive offset filter processor 82′.

The adaptive offset filter information setting unit 81′ includes, asshown in FIG. 30, an offset calculator 811, an offset clipping section812, and an offset information selector 813.

(Offset Calculator 811)

The offset calculator 811 calculates offsets concerning all the offsettypes and all the classes for all QAOUs up to a predetermined splitdepth included in the unit of processing (for example, an LCU) inaccordance with the hierarchical depth of a subject QAOU. In this case,the offset types and the classes are the same as those discussed in adescription of the video decoding device 1.

The offset calculator 811 supplies offset information indicating offsetscalculated by the above-described processing, offset types, classes, andQAOU structure information indicating the QAOU split structure to theoffset clipping section 812.

(Offset Clipping Section 812)

The offset clipping section 812 performs clip processing by using one ofthe following first clip processing and second clip processing onoffsets supplied from the offset calculator 811.

(First Clip Processing)

The offset clipping section 812 clips each offset supplied from theoffset calculator 811 to, for example, values from −8 to 7, therebyexpressing each offset by four bits. The clipped offsets are supplied tothe offset information selector 813. The bit width used for clipping isset in accordance with the image bit depth and the offset bit depth, asin the video decoding device 1.

By clipping each offset in this manner, the memory size of a memory (notshown) to store each offset can be reduced. The amount of data requiredto code offsets included in the coded data #1 can also be decreased,thereby making it possible to improve the coding efficiency.Additionally, since excessive offsets are not added, a suitable level ofimage quality can be guaranteed.

(Second Clip Processing)

The offset clipping section 812 may differently set the clipping rangeof each offset supplied from the offset calculator 811 in accordancewith the offset type.

For example, if the offset type is an edge offset, the number of bits ofan offset is set to be eight bits, and if the offset type is a bandoffset, the number of bits of an offset is set to be four bits. Moregenerally, when the number of bits of an edge-offset offset is N bitsand the number of bits of a band-offset offset is M bits, the numbers ofbits of offsets are determined so that the condition N>M can besatisfied.

In this manner, by varying the number of bits of an offset in accordancewith the offset type, the coding efficiency can be improved withoutrequiring an excessive memory size of a memory for storing each offset.

When the threshold th for restricting values that can be taken as anoffset is greater than 2^(m-1) but not greater than 2^(m), m-bitfixed-length coding method can be used as the coding method for codingthe offset. More specifically, Truncated unary coding or Truncated Ricecoding having th as the maximum value may be used.

Clip processing performed by a combination of the above-described firstclip processing and second clip processing is also included in thisembodiment. The adaptive offset filter 80′ may not include the offsetclipping section 812.

The offset clipping section 812 changes the clipping range in accordancewith the hierarchical depth of a subject QAOU so as to match the offsetprecision. More specifically, a case in which the offset precision isn_(A) bits when the hierarchical depth of a subject QAOU is smaller thana threshold and the offset precision is n_(B) bits when the hierarchicaldepth of a subject QAOU is equal to or greater than the threshold willbe considered. In this case, when the hierarchical depth of a subjectQAOU is smaller than the threshold, the offset clipping section 812 setsthe clipping range to be −2^(nA/2) to 2^(nA/2)−1. When the hierarchicaldepth of a subject QAOU is equal to or greater than the threshold, theoffset clipping section 812 sets the clipping range to be −2^(nA/2) to2^(nA/2)−1, and then sets the lower (n_(A)−n_(B)) bits to be “0”. Thisenables the offset information selector 813 to perform processing byusing offsets subjected to clipping processing performed by the offsetclipping section 812 without considering the offset precision.

(Offset Information Selector 813)

The offset information selector 813 determines a combination of anoffset type, classes, and offsets which minimize the RD cost(Rate-Distortion cost) and the associated QAOU split structure, andsupplies QAOU information indicating the determined offset type,classes, and offsets, and the associated QAOU split structure to thevariable-length code coder 22. The offset information selector 813 alsosupplies the determined offsets for each QAOU to the adaptive offsetfilter processor 82.

Processing performed by the offset information selector 813 will bediscussed more specifically with reference to FIG. 31.

FIG. 31 shows an overview of a case in which a squared error concerninga QAOU having a QAOU index “x” is calculated with respect to each offsettype. As shown in FIG. 31, the offset information selector 813calculates a squared error concerning each QAOMU with respect to eachoffset type. Then, the offset information selector 813 sets the offsettype which minimizes the squared error to be the offset type of theQAOU. As a result, offset types for all the QAOMUs (QAOMU numbers 0 to340) are determined.

Then, the offset information selector 813 calculates the RD cost whenthe split depth is 0 and the RD cost when the split depth is 1. Aspecific calculation method is the same as that discussed with referenceto FIG. 18.

(Adaptive Offset Filter Processor 82′)

The adaptive offset filter processor 82′ adds an offset supplied fromthe offset information selector 813 to each pixel of a subject QAOU inthe deblocked decoded image P_DB. The adaptive offset filter processor82′ outputs, as an offset-filtered decoded image P_OF, an image obtainedby performing processing on all the QAOUs included in the deblockeddecoded image P_DB. The configuration of the adaptive offset filterprocessor 82′ is the same as that of the adaptive offset filterprocessor 62′, and thus, an explanation thereof is omitted here.

Alternatively, an adaptive clip type may be utilized. That is, as one ofoffset types, an adaptive clip (AC) type may be used. In this case,there are three offset types, EO, BO, and AC. In the adaptive clip type(AC), a pixel value is corrected by using clips, such as a lower limitvalue c1 and an upper limit value c2, without using offsets. Examples ofthe lower limit value c1 and the upper limit value c2 are c1=16 andc2=235, respectively.

The use of the adaptive clip type eliminates the need to perform offsetcoding processing and to provide a memory for storing many offsets. Ifthe lower limit value and the upper limit value used for clipping areadaptively provided instead of using fixed values, they are coded. Inthis case, differences from suitable fixed values, for example, anoffset from 16 in the case of the lower limit value and an offset from235 in the case of the upper limit value may preferably be coded. Theupper limit value, particularly, may become a large value. However, thecoding efficiency is not decreased by coding in this manner.

(Appendix 2)

As stated above, “in accordance with the hierarchical depth of a QAOU”may be read as “in accordance with the size of a QAOU”. That is, thenumber of SAO types, the number of SAO classes, and the offset precisionmay be changed in accordance with the size of a QAOU.

For example, if the size of a QAOU is smaller than N×N pixels, one or aplurality of factors, such as the number of SAO types, the number of SAOclasses, and the offset precision, may be restricted more heavily thanthat when the size of a QAOU is equal to or greater than N×N pixels. Aspecific value of N is, for example, N=64 (size of an LCU).

The number of SAO types may be restricted, for example, as follows: EOis restricted to a horizontal or vertical type, BO is restricted to onetype (transform table 802C), only EO is used and BO is not used(transform table 802A), and only horizontal EO and only some modes of BOis used (transform table 801B or 802D). The transform tables indicatedin the parentheses are used in the offset information decoding section611. The restriction of the number of SAO classes and the offsetprecision may be imposed in a manner similar to the above-describedembodiments.

By restricting the modes of SAO types, the number of SAO types, thenumber of SAO classes, and the offset precision in this manner, a memoryspace to be used can be decreased and the processing load can bereduced. In particular, if the modes of SAO types are restricted to BOor if EO is restricted to a horizontal or vertical type, the processingload in the coding device and the classifying section of the decodingdevice can be reduced. If the number of types is restricted, costcalculations for selecting an optimal type can be reduced, therebyreducing the processing load in the coding device. If the modes of SAOtypes are restricted to EO or if the number of classes and the offsetprecision are restricted, a memory space to be used for storing offsetscan be reduced. Moreover, by restricting the modes of the SAO types tohorizontal EO and BO, the number of temporary memories, such as linememories for storing reference pixels used for classifying, can bereduced, and also, a delay occurring while waiting for the decoding ofreference pixels can be decreased. In order to reduce a memory space tobe used, it is more effective if the number of BO classes is restrictedthan the number of EO classes is restricted.

The above-described restrictions can be implemented by the configurationof the means shown in FIGS. 41 through 43 if the operations of theoffset information decoding section 611 and the classifying section 624are adaptively changed in accordance with the size of a QAOU used as aparameter condition.

(Appendix 3)

Additionally, the size of a QAOU to be used when adding an offset to aluminance value (hereinafter also referred to as a “luminance unit”) maybe made different from the size of a QAOU to be used when adding anoffset to a chrominance value (hereinafter also referred to as a“chrominance unit”). Then, the modes of SAO types, the number of SAOtypes, the number of SAO classes, the offset precision, and the maximumsplit hierarchical level may be restricted in a chrominance unit moreheavily than in a luminance unit.

For example, in a data format in which the resolution of the luminanceis different from that of the chrominance and the resolution of thechrominance is lower than that of the luminance, such as in YUV 4:2:0image format, it is not necessary to split a chrominance unit to asmaller level than a luminance unit. Accordingly, by restricting themodes of SAO types, the number of SAO types, the number of SAO classes,and offset precision, and the maximum split hierarchical level of achrominance unit more heavily than those of a luminance unit, a memoryspace to be used can be decreased and the processing load can bereduced. The maximum split hierarchical level of a chrominance unit canbe restricted by making the maximum hierarchical depth of a SAO treestructure smaller than that of a luminance unit.

The above-described restrictions can be implemented by the configurationof the means shown in FIGS. 41 through 43 if the operations of theoffset information decoding section 611 and the classifying section 624are adaptively changed in accordance with the size of a QAOU used as aparameter condition.

In particular, when SAO processing is performed for both of theluminance and the chrominance, a temporary memory, such as a line memoryfor storing reference pixels for classifying, is required for each colorcomponent. Since the luminance component is less important than thechrominance components, the chrominance components are used as aparameter condition, and the modes of SAO types is restricted tohorizontal EO and BO, thereby making it possible to reduce the number ofline memories for the chrominance components. In this case, in theoffset information decoding section 611, a transform table, such as thetransform table 801B or 802D, is utilized.

It is also particularly effective if the chrominance components are usedas a parameter condition and the offset precision for the chrominancecomponents is set to be smaller than that for the luminance component.For example, if the offset precision and the shift value for theluminance component is set to be:SAO_DEPTH=MIN(PIC_DEPTH,AY)shift value=PIC_DEPTH−MIN(PIC_DEPTH,AY); andthe offset precision and the shift value for the chrominance componentsare set to be:SAO_DEPTH=MIN(PIC_DEPTH,THC)shift value=PIC_DEPTH−MIN(PIC_DEPTH,AC),the variables AY and AC for controlling the precision are suitablydetermined such that they may satisfy AY>AC. For example, AY is set tobe 10 or 9, and AC is set to be 8. In this case, the offset precisionfor the luminance component is higher than that for the chrominancecomponents. The shift value for the luminance component is lower thanthat for the chrominance components.(Appendix 4)

Moreover, when the value of a quantization parameter qp (qp value) for aCU is equal to or greater than a threshold, the modes of SAO types, thenumber of SAO types, the number of SAO classes, the offset precision,and the maximum split hierarchical level may be restricted. As the qpvalue, the qp value of the leading portion of a picture may be used, orif a QAOU to be subjected to SAO processing faces a boundary of an LCUor a CU, the qp value corresponding to the top left coordinates or thecentral coordinates of the QAOU may be used.

As the qp value becomes greater, the image quality of a prediction imagebecomes lower, and it is more difficult to perform detailed classifyingprocessing and correction by using offsets. Accordingly, even if themodes of SAO types, the number of SAO types, the number of SAO classes,the offset precision, and the maximum split hierarchical level arerestricted, the influence on the image quality is small. Thus, when theqp value is equal to or greater than a threshold, the number of SAOtypes, the number of SAO classes, the offset precision, and the maximumsplit hierarchical level are restricted. Then, it is possible todecrease a memory space to be used and to reduce the processing loadwithout influencing the image quality.

The above-described restrictions can be implemented by the configurationof the means shown in FIGS. 41 through 43 if the operations of theoffset information decoding section 611 and the classifying section 624are adaptively changed in accordance with QP used as a parametercondition.

(Appendix 5)

Moreover, for a specific picture type, for example, B pictures andnon-reference pictures (IDR pictures), the modes of SAO types, thenumber of SAO types, the number of SAO classes, the offset precision,and the maximum split hierarchical level may be restricted.

Concerning I pictures and P pictures, the image quality of thesepictures greatly influence the subsequent pictures, and thus, it isnecessary to maintain the SAO precision at a high level. On the otherhand, pictures other than I pictures and P pictures do not greatlyinfluence the subsequent pictures, and thus, concerning these pictures(B pictures and IDR pictures), the modes of SAO types, the number of SAOtypes, the number of SAO classes, the offset precision, and the maximumsplit hierarchical level may be restricted. Then, it is possible todecrease a memory space to be used and to reduce the processing load.

The above-described restrictions can be implemented by the configurationof the means shown in FIGS. 41 through 43 if the operations of theoffset information decoding section 611 and the classifying section 624are adaptively changed in accordance with a picture type used as aparameter condition.

(Appendix 6)

Additionally, in accordance with the position of a QAOU on a screen, thenumber of SAO types, the number of SAO classes, the offset precision,and the maximum split hierarchical level may be restricted.

For example, at the periphery of a screen, the number of SAO types, thenumber of SAO classes, the offset precision, and the maximum splithierarchical level may be restricted.

A user is likely to focus on and around the center of the screen, andthus, a decrease in the image quality at this portion directly reflectsthe subjective image quality. However, even if the image quality aroundthe periphery of the screen is decreased, the subject image quality isnot decreased to the same level as that when it is decreased on andaround the center of the screen.

Additionally, if a SAO type which requires sample points beyond the edgeof a screen is not used, the load in boundary determining processing canbe reduced.

The above-described restrictions can be implemented by the configurationof the means shown in FIGS. 41 through 43 if the operations of theoffset information decoding section 611 and the classifying section 624are adaptively changed in accordance with the position of a QAOU used asa parameter condition.

Fourth Embodiment

Another embodiment of the present invention will be described below withreference to FIG. 32. A fourth embodiment is different from theabove-described embodiments in that EO and BO are switched in accordancewith a pixel value.

The present inventors have found that there are characteristics in whichEO is more effective than BO in an intermediate tone area of pixels andBO is more effective than EO in the other areas (a lower pixel valuearea and a higher pixel value area). Accordingly, in this embodiment, BOis used in the lower pixel value area and the higher pixel value areaand EO is used in the intermediate tone area.

This will be described below more specifically with reference to FIG.32. FIG. 32 illustrates the configuration in which EO and BO areswitched in accordance with the pixel value: part (a) shows an overviewof the configuration in which EO and BO are switched in accordance withthe pixel value; part (b) shows specific values to be switched; and part(c) shows the content of a list memory stored in the offset informationstorage section 621.

As shown in part (a) of FIG. 32, in this embodiment, BO is used when thepixel value (luminance value) is around 0 or 255, and EO is used whenthe pixel value is in the other areas. That is, the SAO type is changeddepending on the pixel value, and when the pixel values are around 0 or255, an offset is added in accordance with the pixel value of a pixel,and when the pixel value is in the other areas, an offset is added inaccordance with the type of edge.

It has been found through the inventors' experiments that there areparticular ranges of pixel values to be used both for EO and BO. Thatis, EO is more frequently used in the intermediate tone area. This meansthat, in the lower pixel value area and in the higher pixel value area,pixel values influence errors more greatly than the type of edge.

Accordingly, in this embodiment, EO and BO are switched in accordancewith the pixel value range, and in one SAO type, EO or BO having ahigher error correcting effect is utilized, thereby making it possibleto improve the coding efficiency. Additionally, since it is possible todecrease the number of SAO types and the total number of classes, amemory space to be used can be decreased and the processing load canalso be reduced.

This embodiment will be described below in a greater detail. Unlike theabove-described embodiments, in this embodiment, the following four SAOtypes, “sao_type_idx”=1 to “sao_type_idx”=4, are used and are defined asfollows.

-   -   “sao_type_idx”=1: (BO_EO_0) corresponding to EO_0+BO in the        first embodiment    -   “sao_type_idx”=2: (BO_EO_1) corresponding to EO_1+BO in the        first embodiment    -   “sao_type_idx”=3: (BO_EO_2) corresponding to EO_2+BO in the        first embodiment    -   “sao_type_idx”=4: (BO_EO_3) corresponding to EO_3+BO in the        first embodiment EO_0 to 3 and BO_0, 1 in the first embodiment        are not used.

Then, as shown in part (b) of FIG. 32, the classifying section 624performs classifying by using BoTbl[BO_1] in the transform table 1301shown in FIG. 29. When the pixel value is ¼ of max or smaller or is ¾ ofmax or greater, the classifying section 624 performs classifying byusing the transform table 1301.

When the pixel value is between ¼ and ¾ of max, the classifying section624 determines the edge type in accordance with EO_0 to 3 discussed inthe above-described embodiments, and performs classifying.

With this operation, the classifying section 624 is able to performclassifying by switching between BO and EO in accordance with the pixelvalue.

The offset calculator 811 calculates offsets in a manner similar to theoffset calculating method discussed in the above-described embodiments.However, in this embodiment, the number of modes of offset types isfour, and the offset calculator 811 calculates offsets by changing thenumber of modes of offset types to four.

In a list memory 2101 stored in the offset information storage section621, as shown in part (c) of FIG. 32, the QAOU index, the offset type,and specific values of offsets with respect to classes that can beselected for this offset type are stored in association with each other.

When an offset type having characteristics of EO and BO is used,classifying may be first performed by using EO, and then, for someclasses (classes having a flat edge), classifying may be furtherperformed by using BO, or both of EO and BO may be used, as discussed inthis embodiment. Alternatively, it may be determined depending on thecondition discussed in the above-described embodiments (depending onwhether the hierarchical depth of a QAOU is smaller than a threshold)whether an offset type having characteristics of EO and BO will beprovided. In this case, the number of types and the number of classescan be reduced.

Fifth Embodiment

Another embodiment of the present invention will be described below withreference to FIGS. 33, 34, and 42. A fifth embodiment is different fromthe above-described embodiments in that when performing classifyingusing EO, pixels used for determining the type of edge are restricted topixels positioned in the horizontal direction of a subject pixel.

In the above-described embodiments, a pixel positioned in the upward ordownward direction of a subject pixel is also used for determining thetype of edge. Accordingly, for a pixel (pixel subjected to offsetprocessing) positioned in the upward direction of the subject pixel, aline buffer for storing the pixel value of this pixel which has not beensubjected to offset processing is necessary.

Accordingly, by restricting pixels used for determining the type of edgeto pixels positioned in the horizontal direction of a subject pixel, noreference is made to pixels in the upward direction, thereby making itpossible to reduce a memory space by an amount equal to a line buffer.Moreover, processing for determining a boundary in the upward direction(screen edge) is not necessary, thereby making it possible to reduce theamount of processing.

Additionally, if pixels positioned only in the horizontal direction areused, the processing results which have been obtained so far can beutilized, thereby making it possible to further reduce the amount ofprocessing. This will be discussed below with reference to part (a) ofFIG. 33. Part (a) of FIG. 33 shows pixels positioned in the horizontaldirection, and x0 through x3 indicate pixel values of pixels.

Concerning the pixel having the pixel value x1, the difference betweenthis pixel and the two adjacent pixels are as follows.s1=sign(x1−x0)−sign(x2−x1)

Then, the difference between the pixel having the pixel value x2 and thetwo adjacent pixels are as follows.s2=sign(x2−x1)−sign(x3−x2)

In the above-described expressions, s1 and s2 are values used forclassifying an edge.

In this manner, when calculating s2, sign(x2−x1) used for calculating s1is reused. Accordingly, the amount of processing can be reduced by theamount of this reuse.

Details of this embodiment will be discussed below. In this embodiment,the EO type is restricted to two modes. That is, “sao_type_idx”=1, 2 areused as edge offsets (EO). When “sao_type_idx”=1 (EO_0), the type ofedge is determined by comparing a subject pixel with pixels positionedimmediately on the right and left sides of the subject pixel (part (b)of FIG. 33). When “sao_type_idx”=2 (EO′_0), the type of edge isdetermined by comparing a subject pixel with pixels positioned twopixels away from the subject pixel on the right and left sides (part (c)of FIG. 33).

When “sao_type_idx”=3, 4, a band offset (BO) is used. In this case,“sao_type_idx”=5, 6 in the first embodiment may be read as“sao_type_idx”=3, 4, respectively.

The purpose of determining the type of edge by using pixels positionedtwo pixels away from a subject pixel when “sao_type_idx”=2 is tofacilitate the detection of an edge having a small angle with respect tothe horizontal direction. As shown in part (d) of FIG. 33, if two pixelsimmediately next to a subject pixel are used for comparison, thedifference between the pixel value of the subject pixel and those ofsuch adjacent pixels is small, and may not be judged as an edge. Even inthis case, if the subject pixel is also compared with pixels positionedtwo pixels away from the subject pixel, an edge having a small anglewith respect to the horizontal direction can be detected.

Instead of using two pixels immediately next to a subject pixel, a pixelpositioned immediately on the left side of a subject pixel and a pixelpositioned two pixels away from the subject pixel on the right side(part (a) of FIG. 34) may be used for determining the difference.Conversely, a pixel positioned immediately on the right side of thesubject pixel and a pixel positioned two pixels away from the subjectpixel on the left side (part (b) of FIG. 34) may be used for determiningthe difference. These cases are particularly effective when referencepixels are positioned at an edge of a screen.

The use of pixels positioned only in the horizontal direction may berestricted to some cases. For example, in the case of pictures whichinfluence other pictures less, such as B pictures and non-referencepictures, reference may be made only to pixels in the horizontaldirection. In the case of other pictures, reference may be made topixels in a manner similar to the first embodiment. Alternatively, at anedge of a screen or near a boundary of a slice, reference may be madeonly to pixels in the horizontal direction, and at the other areas,reference may be made to pixels in a manner similar to the firstembodiment. With this arrangement, a memory space to be used can bedecreased, and the amount of processing can also be reduced.Additionally, since no reference is made in the upward direction, theamount of boundary determining processing for determining boundaries,such as an edge of a screen or a boundary of a slice, can also bereduced.

Pixels to be used may be explicitly specified by using a flag for eachpicture or each block.

For example, if there are many horizontal edges (part (c) of FIG. 34),processing is performed by using the method discussed in theabove-described embodiments, thereby suppressing a decrease in theperformance.

Alternatively, it may be determined, depending on the conditiondiscussed in the above-described embodiments (depending on whether thehierarchical depth of a QAOU is smaller than a threshold), whetherpixels positioned only in the horizontal direction will be used.Moreover, only when BO and EO are both used, as discussed in theabove-described embodiment, may pixels positioned only in the horizontaldirection be used for EO.

The type of edge may be determined depending on whether the differencebetween a subject pixel and a reference pixel positioned in thehorizontal direction of the subject pixel is greater (or smaller) than athreshold. More specifically, classifying may be performed according tothe following equations.Sign(z)=+1 (when z>th)Sign(z)=0 (when −th≤z≤th)Sign(z)=−1 (when z<−th)In the these equations, th denotes a threshold having a predeterminedvalue.

In the case of a pixel value indicating a chrominance component, pixelspositioned only in the horizontal direction may be used for an edgeoffset.

The effect obtained by restricting pixels to those in the horizontaldirection may be achieved even when only one horizontal-edge classifyingmethod is provided, for example, even when the method shown in part (b)of FIG. 33 is used. If, as described above, a plurality of differenthorizontal-edge classifying methods are combined, more precise edgeclassifying can be implemented than that when the method shown in part(b) of FIG. 33 is singly used. A combination of a plurality ofhorizontal-edge classifying methods is not restricted to the use of twoexamples by changing the distance between a subject pixel and referencepixels, as in parts (b) and (c) of FIG. 33, but may be the use of twothresholds for determining the type of edge. For example, twohorizontal-edge classifying methods, that is, one method when thethreshold th is 0 and another method when the threshold th is 1, may beused.

The configuration in which the use of pixels positioned only in thehorizontal direction is restricted to some cases can be implemented bythe configuration of the means shown in FIGS. 41 through 43 of the firstembodiment if the operations of the offset information decoding section611 and the classifying section 624 are adaptively changed in accordancewith a parameter condition. For example, in the offset informationdecoding section 611 shown in FIG. 41, the modes of offset types to beselected in the offset type selector 6112 in accordance with a parametercondition are restricted only to the horizontal-edge classifyingmethods, and the restricted modes of the offset types are decoded in theoffset type decoder 6113.

As shown in part (b) of FIG. 42, the offset type transform tableselector 6117′ selects the transform table stored in theedge-offset-type-and-band-offset-type transform table storage section6118′ or the transform table stored in thehorizontal-edge-offset-type-and-band-offset-type transform table storagesection 6119′ in accordance with the parameter condition. In oneexample, if the color component is a luminance component, the offsettype transform table selector 6117 selects the transform table stored inthe edge-offset-type-and-band-offset-type transform table storagesection 6118′. If the color component is a chrominance component, theoffset type transform table selector 6117 selects the transform tablestored in the horizontal-edge-offset-type-and-band-offset-type transformtable storage section 6119′.

The offset type selector 6112 selects an edge classifying method whichis restricted to a horizontal edge classifying method in accordance withthe parameter condition, and, in the classifying section 624, the edgeoffset classifying section 6243 classifies pixels. In one example, ifthe color component is a luminance component, the basic edge classifyingmethod is selected, and if the color component is a chrominancecomponent, a horizontal edge classifying method is selected.

Sixth Embodiment

Another embodiment of the present invention will be described below withreference to FIGS. 35 through 37. A sixth embodiment is different fromthe above-described embodiments in that the offset precision is improvedor more precise classifying is performed around the center of the pixelvalues of chrominance components.

An image has characteristics in which errors around an achromatic colorare noticeable in terms of the subjective image quality. In the pixelvalues of each chrominance component, an achromatic color is positionedat the center of the value range, and if the bit depth is eight bits,the achromatic color is a color having a pixel value of 128.Accordingly, in this embodiment, in the case of BO, around the pixelvalue of 128 of a chrominance component, the offset precision isimproved or more precise classifying is performed by using a smallerclass width. Concerning the pixel values of a chrominance component, thevalue range around an achromatic color is referred to as the “achromaticcolor value range”.

With this arrangement, since the offset precision can be improved in theachromatic color value range, the performance in making corrections byusing offsets in the achromatic color value range can be enhanced.Accordingly, errors caused by the deviation from the original image inthe achromatic color value range are decreased, thereby making itpossible to improve the subjective image quality. Moreover, byperforming more precise classifying using a smaller class width in theachromatic color value range than in other value ranges, correctionsusing offsets in the achromatic color value range can be performedprecisely. Accordingly, errors caused by the deviation from the originalimage in the achromatic color value range are decreased, thereby makingit possible to improve the subjective image quality.

In this embodiment, the offset precision is improved or more preciseclassifying is performed for the achromatic color value range. However,the value range subjected to this processing is not restricted to theachromatic color value range. More precise classifying may be performedor the offset precision may be improved for a value range which islikely to influence the subjective image quality, thereby making itpossible to improve the subjective image quality. Additionally, for thepixel value of a luminance component, as well as that of a chrominancecomponent, for a value range in which degradation of the image qualityis likely to be noticeable in terms of the subjective image quality, theoffset precision may be similarly improved or more precise classifyingmay be similarly performed, thereby achieving advantages similar tothose obtained for a chrominance component.

A case in which the offset precision is improved will first be discussedbelow with reference to FIG. 35. FIG. 35 illustrates an overview of acase in which the offset precision is improved. In order to improve theoffset precision, as shown in FIG. 35, the offset precision around thepixel value of 128 of a chrominance component is set to be high, and theoffset precision in the other value ranges is set to be low.

The offset information storage section 621 secures a storage area ofn_(B) bits (for example, n_(B)=8) for offsets used for a value rangehaving a higher offset precision, and secures a storage area of n_(A)bits (for example, n_(A)=6) for Offsets used for a value range having alower offset precision. If there is an enough storage area, the storagearea may be set to be unconditionally n_(B) bits, in which case, theimplementation is facilitated.

The offset clipping section 812 utilizes different clipping ranges for avalue range having a high offset precision and a value range having alow offset precision. For example, for a value range having a low offsetprecision, the clipping range is set to be −2^(nA/2) to 2^(nA/2)−1, andfor a value range having a high offset precision, the clipping range isset to be −2^(nB/2) to 2^(nB/2)−1.

A case in which more precise classifying is performed will now bediscussed below with reference to FIG. 36. FIG. 36 illustrates anoverview of a case in which more precise classifying is performed: part(a) shows a transform table; and parts (b) through (d) show that pixelvalues are divided.

When more precise classifying is performed, in a list memory for achrominance component in the offset information storage section 621, anoffset storage area corresponding to “sao_type_idx”=5 is secured inaccordance with the number of classifying levels.

In the case of BO, the classifying section 624 considers whether or notthe pixel value of a subject pixel is contained within the achromaticcolor value range when determining a class from the pixel value. Morespecifically, the classifying section 624 performs classifying by usingone of the patterns in a transform table 2501 shown in part (a) of FIG.36.

In the transform table 2501, classifying is performed by using“class_idx=BoTbl[sao_type_idx][pic[x]/4]”, and classes are assigned tosmaller widths of pixel values than those in the transform tables 1301and 1302 shown in FIG. 29.

“BoTbl[BO_0][pix/4](a)” in the transform table 2501 shows an example(part (b) of FIG. 36) in which there are sixteen classes, and theachromatic color value range is split into smaller widths and theintermediate tone areas (pix/4=16 to 21, 42 to 47) are split into largerwidths. “BoTbl[BO_0][pix/4](b)” shows an example (part (c) of FIG. 36)in which there are sixteen classes, and the range in which offsets areadded is narrowed, and the achromatic color value range is split intosmaller widths while the maximum number of classes and the maximum classwidth are maintained. “BoTbl[BO_0][pix/4](c)” shows an example (part (d)of FIG. 36) in which there are eighteen classes, and the achromaticcolor value range is split into smaller widths while the maximum classwidth is maintained. Additionally, in another example, the class widthof the intermediate tone areas is set to be relatively larger than thatof the achromatic color value range, thereby decreasing the number ofentire classes. In this example, it is possible to reduce the space ofthe offset storage area while maintaining the correction precision inthe achromatic color value range. Such an arrangement is possiblebecause the frequency with which offsets are used for correction in theintermediate tone areas is lower than that in the achromatic color valuerange.

Alternatively, both of an improvement in the offset precision and moreprecise classifying in the achromatic color value range may beperformed.

Alternatively, the offset precision may be changed according to whetherthe pixel value indicates a luminance component or a chrominancecomponent. That is, the offset precision for the pixel value indicatinga chrominance component may be set to be lower than that for the pixelvalue indicating a luminance component.

(Appendix 7)

The addition of an offset may be performed such that the pixel valueapproximates an achromatic color (=pixel value of 128 of a chrominancecomponent). That is, the offset adder 626 may handle offsets such thatthe sign of an offset to be added to the pixel value of 128 or greateris opposite to the sign of an offset to be added to the pixel valuesmaller than 128. For example, when an offset is indicated by a, thesign of the offset may be determined such that x′=x+a (x<128) andx′=x−a(x≥128).

If 128 is positioned between the pixel value to which an offset has notbeen added and the pixel value to which an offset has been added, thepixel value to which an offset has been added may be clipped by 128. Forexample, if the pixel value is “126 (blue)” and the offset is “3”,instead of directly adding the offset to the pixel value such thatx=126+3=129, x may be set to x-clip(126+3, 0, 128)=128. In thisequation, clip(x, y, z) indicates processing for restricting the valueof x to y≤x≤z.

If the pixel value is “129 (red)” and the offset is “−4”, instead ofdirectly adding the offset to the pixel value such that x=129+(−4)=125,x may be set to x=clip(129+(−4), 128, 255)=128.

(Appendix 8)

For a chrominance component, offsets may be added in one of theintermediate tone areas and the other areas. For example, as shown inpart (a) of FIG. 37, classifying may be performed by focusing on thepixel values of an achromatic color. In the example shown in part (a) ofFIG. 37, in the case of “sao_type_idx”=5(BO_0), classes are assigned tothe intermediate tone areas, and in the case of “sao_type_idx”=6(BO_1),classes are assigned to value ranges other than the intermediate toneareas.

Alternatively, as shown in part (b) of FIG. 37, classifying may beperformed asymmetrically in two value ranges with respect to the pixelvalues of an achromatic color. In the example shown in part (b) of FIG.37, the class width is the same as that of part (a) of FIG. 37, andpartitioning positions of the classes are different from those of part(a) of FIG. 37.

Alternatively, as shown in parts (c) and (d) of FIG. 37, classifying maybe performed differently in accordance with the chrominance channel (Cror Cb). In the examples shown in parts (c) and (d) of FIG. 37, the classwidth is different between the chrominance component (Cr) and thechrominance component (Cb). The class width can be changed by settingmore precise classifying levels, as in the transform table 2501.

As shown in part (e) of FIG. 37, for the chrominance components,classifying may be performed by using only one BO type.

Seventh Embodiment

Another embodiment of the present invention will be described below withreference to FIGS. 38 through 40. A seventh embodiment is different fromthe above-described embodiments in that, instead of using the value ofan offset itself, the value of an offset is subjected to predictivecoding, that is, an offset residual calculated by using the value of anoffset and a prediction value of the offset value is coded and in that“0” may be provided as the prediction value of an offset.

Generally, in SAO, an offset is coded in accordance with a classindicating a classified area. If there is no pixel classified under acertain class (class is empty), “0” is coded. Generally, offsets inadjacent classes or adjacent areas have similar values. Accordingly, ifan offset value which has already been decoded is used as a predictionvalue of another offset and the difference from this prediction value iscoded, the amount of data to code offsets can be reduced. However, ifthe class is empty, the amount of data to code offsets is increased.

In this embodiment, therefore, “0” is included as prediction values ofoffsets, and one prediction value is selected from a plurality ofcandidates of prediction values including “0”, and then, the predictiondifference is coded, thereby reducing the amount of data to codeoffsets.

With this arrangement, regardless of whether a class is empty or thereis a pixel classified under this class and an offset is coded, asuitable prediction value can be assigned, thereby making it possible toreduce the amount of data to code offsets by performing predictivecoding.

The configuration of an offset information decoding section 611′ of thisembodiment will be described below with reference to FIG. 38. The offsetinformation decoding section 611′ is provided instead of the offsetinformation decoding section 611 shown in FIG. 22, and includes anoffset residual decoder 651, an offset reconstructing section 652, and aprediction value determining section 653. The prediction valuedetermining section 653 includes a prediction candidate flag decoder661, a fixed prediction value calculator 662, a coded prediction valuecalculator 663, and a prediction value candidate selector 664.

The offset residual decoder 651 decodes an offset residual from QAOUinformation included in coded data #1 and supplies the decoded offsetresidual to the offset reconstructing section 652.

The prediction value determining section 653 determines a predictionvalue of an offset. The prediction candidate flag decoder 661 decodes aprediction candidate flag from the QAOU information and supplies thedecoded prediction candidate flag to the prediction value candidateselector 664. The prediction candidate flag decoder 661 may beconfigured such that it does not decode the candidate flag if apreviously decoded offset value is “0”.

The fixed prediction value calculator 662 calculates a fixed value (inthis case, “0”), which is an offset to be coded when there is no pixelclassified under a certain class, as a prediction value, and suppliesthe calculated fixed value to the prediction value candidate selector664.

The coded prediction value calculator 663 reads a decoded offset fromthe offset information storage section 621 and calculates a predictionvalue, and supplies it to the prediction value candidate selector 664.

The prediction value candidate selector 664 selects, in accordance withthe prediction candidate flag, a prediction value to be supplied to theoffset reconstructing section 652 from among the prediction valuesupplied from the fixed prediction value calculator 662 and theprediction value supplied from the coded prediction value calculator663.

The offset reconstructing section 652 reconstructs an offset (Offset)from the prediction value (pred) supplied from the prediction valuecandidate selector 664 and from the offset residual (sao_offset)supplied from the offset residual decoder 651 according to the followingequation.“Offset”=“pred”+“sao_offset”

The reason why the amount of data required for coding is increased ifthere is a class (empty class) under which no pixel is classified willbe discussed below with reference to FIG. 39. FIG. 39 illustrates anoverview of a case in which there are empty classes. As shown in FIG.39, in a class which is not empty, an offset used for this class iscoded. Then, for each class, by using the previous offset as aprediction value, the difference between the prediction value and thecurrent offset is coded. However, if there is an empty class, theprediction value becomes “0”, and the difference value is increased. Asa result, the amount of data required for coding is increased.

In this embodiment, therefore, by using a prediction candidate flag, theprediction value may be selected from “0” and “previous offset (otherthan 0)”.

Syntax to be used when the prediction candidate flag is used will bedescribed below with reference to FIG. 40. FIG. 40 illustrates syntax2901 to be used when a prediction candidate flag is used. As shown inFIG. 40, a prediction candidate flag “sao_pred_flag” is added, and if“sao_pred_flag”=0, the previously decoded offset (other than 0) is setas a prediction value offsetp. If “sao_pred_flag”=1 or the currentoffset is the first offset, the prediction value offsetp is set to be 0.In the syntax 2901, “sao_offset_delta” represents a difference d fromthe prediction value, and“sao_offset[sao_curr_depth][ys][xs]”=“offsetp+d”.

The prediction candidate flag may not be used depending on the offsettype. For example, in the case of EO, the prediction candidate flag isnot used, and in the case of BO, the prediction candidate flag is used.This is because, in BO, empty classes may be frequently generateddepending on the distribution of pixel values.

Additionally, a determination may be made whether to perform predictivecoding itself depending on the offset type. For example, in the case ofEO, predictive coding may be performed, and in the case of BO,predictive coding may not be performed.

(Appendix 9)

In the above-described third through seventh embodiments, by using theintra-prediction mode, edge offset classes may be estimated, restricted,rearranged for each LCU.

An edge direction selected as an EO type of SAO may have correlationwith an intra-prediction mode. Accordingly, reference to theintra-prediction mode of a CU located at a position corresponding to aQAOU of SAO can be utilized for selecting an EO class.

For example, by using the intra-prediction mode, an EO class may beestimated and determined, EO class candidates may be restricted, or theorder (index) of EO classes may be rearranged in order in which they aremore likely to be selected.

This is particularly easy to implement if an image is divided such thatthe size of a QAOU of each hierarchical level of SAO is the same as thatof a CU, for example, such that the maximum size of a QAOU of SAO isequal to the size of an LCU.

(Appendix 10)

In a known adaptive offset filter, since many modes of offsets (typesand classes) are provided, a large memory size may be required.Accordingly, in this embodiment, for example, an image filtering devicewhich is capable of reducing block distortion while suppressing anincrease in the memory size is implemented.

An image filtering device according to the present invention is an imagefiltering device for adding an offset selected from among a plurality ofoffsets to a pixel value of each pixel forming an input image which isconstituted by a plurality of unit areas. The image filtering deviceincludes: offset determining means for determining, for each unit area,an offset to be added to the pixel value of each pixel included in theunit area; and filtering means for adding an offset determined by theoffset determining means to the pixel value of each pixel included inthe unit area. In a case in which the size of a unit area for which anoffset is determined is smaller than a predetermined size, the offsetdetermining means determines an offset to be added to the pixel value ofeach pixel included in the unit area from among a more restricted numberof selectable offsets than that in a case in which the size of the unitarea is equal to or greater than the predetermined size.

With the above-described configuration, in a case in which the size of aunit area for which an offset is determined is smaller than thepredetermined size, an offset to be added is determined from among amore restricted number of offsets than that in a case in which the sizeof the unit area is equal to or greater than the predetermined size.

When the size of a unit area is small, the number of pixels included inthis unit area is also small, and it is more likely that pixels in thisunit area have similar values. Accordingly, when the size of a unit areais small, even if the number of selectable offsets is restricted, theinfluence on an image to which an offset is applied is small.Additionally, by restricting the number of selectable offsets, arequired memory space can be reduced.

Thus, with the above-described configuration, it is possible to reduce amemory space to be used while the influence on an image to which anoffset is applied is decreased. Additionally, since the number ofselectable offsets is decreased, the amount of data to code offsets canbe reduced, thereby improving the coding efficiency.

In order to solve the above-described problem, an image filtering deviceaccording to the present invention is an image filtering device foradding an offset to a pixel value of each pixel forming an input imagewhich is constituted by a plurality of unit areas. The image filteringdevice includes: offset determining means for determining, for each unitarea, an offset to be added to the pixel value of each pixel included inthe unit area; and filtering means for adding an offset determined bythe offset determining means to the pixel value of each pixel includedin the unit area. In a case in which the size of a unit area for whichan offset is determined is smaller than a predetermined size, thefiltering means adds an offset having a lower precision than that in acase in which the size of the unit area is equal to or greater than thepredetermined size.

With the above-described configuration, in a case in which the size of aunit area for which an offset is determined is smaller than thepredetermined size, an offset having a lower precision than that in acase in which the size of the unit area is equal to or greater than thepredetermined size is added.

When the size of a unit area is small, the number of pixels included inthis unit area is also small, and it is more likely that pixels in thisunit area have similar values. Accordingly, when the size of a unit areais small, even if the precision of an offset is decreased, the influenceon quantization errors is small, and thus, the influence on an image towhich an offset is applied is also small. Additionally, by decreasingthe precision of an offset, a required memory space can be reduced.

Thus, with the above-described configuration, it is possible to reduce amemory space to be used while the influence on an image to which anoffset is applied is decreased. Additionally, since the precision of anoffset is decreased, the amount of data to code offsets can be reduced,thereby improving the coding efficiency.

An image filtering device according to the present invention is an imagefiltering device for performing adaptive offset (SAO: Sample AdaptiveOffset) on a pixel value of each pixel forming an input image which isconstituted by a plurality of unit areas. The image filtering deviceincludes: offset determining means for determining, for each unit area,an offset type of offset to be added to the pixel value of each pixelincluded in the unit area; and filtering means for adding an offset ofan offset type determined by the offset determining means to the pixelvalue of each pixel included in the unit area. The offset determiningmeans determines the offset type to be a band offset (BO) in a case inwhich the pixel value of a subject pixel to which an offset is added ispositioned in a value range around a maximum value or a minimum value,and determines the offset type to be an edge offset (EO) in a case inwhich the pixel value of the subject pixel is positioned in a valuerange other than the value ranges around the maximum value and theminimum value.

With the above-described configuration, when adaptive offsetting isperformed, a band offset is applied to pixels having pixel valuespositioned around the maximum value or the minimum value, and an edgeoffset is applied to pixels having pixel values positioned in valueranges other than the value ranges of the maximum value and the minimumvalue.

In a higher pixel value range or a lower pixel value range, it is morelikely that pixel values influence errors than edges do. Accordingly,with the above-described configuration, the efficiency in correctingerrors can be enhanced, thereby improving the coding efficiency.

If one offset type includes both of a band offset and an edge offset,the number of types of adaptive offsets can be decreased, therebydecreasing a memory space to be used and also reducing the amount ofprocessing.

In the above-described configuration, band offset processing is offsetprocessing for adding one of a plurality of offsets to the pixel valueof a subject pixel in accordance with the magnitude of the pixel valueof the subject pixel (this also applies to the subsequent description).Edge offset processing is offset processing for adding one of aplurality of offsets to the pixel value of a subject pixel in accordancewith the difference between the pixel value of the subject pixel and thepixel value of a pixel positioned around the subject pixel (this alsoapplies to the subsequent description).

In the image filtering device according to the present invention, theoffset determining means may determine the offset type to be a bandoffset for pixels positioned in a value range from the minimum value to¼ of the maximum value or a value range from ¾ of the maximum value tothe maximum value, and may determine the offset type to be an edgeoffset for pixels positioned in value ranges other than these valueranges.

With the above-described configuration, it is possible to clearlyseparate pixels to which a band offset is applied from pixels to whichan edge offset is applied.

An image filtering device according to the present invention is an imagefiltering device for performing adaptive offset (SAO: Sample AdaptiveOffset) on an input image. The image filtering device includes:classifying means for determining an edge in order to determine a classto be used in an edge offset (EO) by referring to pixels positioned onlyin the horizontal direction of a subject pixel; and filtering means foradding an offset associated with a class determined by the classifyingmeans in a case in which an edge offset (EO) is applied.

With the above-described configuration, when determining an edge,reference is made to pixels positioned only in the horizontal direction,thereby making it possible to reduce a required memory space, comparedwith a case in which reference is also made to pixels in the upwarddirection. Additionally, boundary determining processing for determiningboundaries in the upward direction is not necessary, thereby making itpossible to reduce the amount of processing.

In the image filtering device according to the present invention, theclassifying means may determine an edge by referring to a pixelpositioned two pixels away from the subject pixel in the horizontaldirection.

With the above-described configuration, since reference is made to apixel positioned two pixels away from the subject pixel, it is evenpossible to detect an edge having a small angle.

An image filtering device according to the present invention is an imagefiltering device for performing adaptive offset (SAO: Sample AdaptiveOffset) on an input image. The image filtering device includes:classifying means for determining a class to be used in a band offset(BO) by setting the division width of classes to be used in a range ofpixel values indicating the chrominance around a center value, which isa value positioned at the center between the maximum value and theminimum value, to be smaller than that in the other ranges of the pixelvalues; and filtering means for adding an offset associated with a classdetermined by the classifying means in a case in which a band offset(BO) is applied.

With the above-described configuration, the division width of classes tobe used in a range of pixel values indicating the chrominance around thecenter value, which is a value positioned at the center between themaximum value and the minimum value, is set to be smaller than that inthe other ranges of the pixel values, and then, the classes to be usedwhen a band offset (BO) is applied are determined.

When the pixel value indicating the chrominance is the center value,this pixel is an achromatic color. Errors of an achromatic color arenoticeable to the human eye, thereby decreasing the subjective imagequality. Then, as in the above-described configuration, if the classwidth in a range around the center value is set to be small, offsets canbe set precisely for pixels around the center value. As a result, thesubjective image quality can be increased.

In the image filtering device according to the present invention, if thecenter value is positioned between the pixel value to which an offsethas not been added and the pixel value to which an offset has beenadded, the filtering means may set the pixel value to which an offsethas been added to be the center value.

With the above-described configuration, an offset is not added if thepixel value to which an offset has been added is beyond the centervalue. When the pixel value indicating the chrominance is the centervalue, this pixel is an achromatic color, and in value ranges with thecenter value therebetween, the color perceived by the human eye ischanged from the achromatic color. Then, with the above-describedconfiguration, it is possible to prevent the color perceived by thehuman eye from changing, which would otherwise be caused by adding anoffset.

An image filtering device according to the present invention is an imagefiltering device for performing adaptive offset (SAO: Sample AdaptiveOffset) on an input image. The image filtering device includes:filtering means for adding an offset having a higher precision to apixel having a pixel value indicating the chrominance around a centervalue, which is a value positioned at the center between the maximumvalue and the minimum value, than the precision of an offset to be addedto a pixel in the other value ranges.

With the above-described configuration, an offset having a higherprecision is added to a pixel having a pixel value indicating thechrominance around a center value, which is a value positioned at thecenter between the maximum value and the minimum value, than theprecision of an offset to be added to a pixel in the other value ranges.

When the pixel value of a pixel indicating the chrominance is the centervalue, this pixel is an achromatic color. Errors of an achromatic colorare noticeable to the human eye, thereby decreasing the subjective imagequality. Then, as in the above-described configuration, if the precisionof an offset to be added to pixels around a center value is improved,offsets can be precisely added to pixels around the center value. As aresult, the subjective image quality can be increased.

An offset decoding device according to the present invention is anoffset decoding device for decoding each offset which is referred to byan image filter for adding an offset to a pixel value of each pixelforming an input image. The offset decoding device includes: offsetresidual decoding means for decoding each offset residual from codeddata; prediction value determining means for determining a predictionvalue of each offset from a decoded offset or a predetermined value; andoffset calculating means for calculating each offset from a predictionvalue determined by the prediction value determining means and an offsetresidual decoded by the offset residual decoding means.

With the above-described configuration, since an offset is decoded froma residual, the amount of data to code an offset can be decreased,compared with a case in which each offset itself is coded. Additionally,a prediction value for determining a residual is determined from adecoded offset or a predetermined value. Thus, it is possible to preventthe amount of data to code difference data from becoming greater thanthat when an offset itself is coded, which would otherwise be caused bythe use of decoded offsets only.

The predetermined value may be, for example, “0”.

An offset coding device according to the present invention is an offsetcoding device for coding each offset which is referred to by an imagefilter for adding an offset to a pixel value of each pixel forming aninput image. The offset coding device includes: prediction valuedetermining means for determining a prediction value of each offset froma coded offset or a predetermined value; offset residual calculatingmeans for calculating an offset residual from each offset and aprediction value determined by the prediction value determining means;and offset residual coding means for coding an offset residualcalculated by the offset residual calculating means.

With the above-described configuration, since an offset is decoded froma residual, the amount of data to code an offset can be decreased,compared with a case in which each offset itself is coded. Additionally,a prediction value for determining a residual is determined from adecoded offset or a predetermined value. Thus, it is possible to preventthe amount of data to code difference data from becoming greater thanthat when an offset itself is coded, which would otherwise be caused bythe use of decoded offsets only.

A data structure of coded data according to the present invention is adata structure of coded data which is referred to by an image filter foradding an offset to a pixel value of each pixel forming an input imagewhich is constituted by a plurality of unit areas. The data structureincludes: prediction value determining information indicating whether aprediction value will be determined from a decoded offset or apredetermined value. The image filter refers to the prediction valuedetermining information included in the coded data, and then determinesa prediction value and decodes an offset.

With the above-described configuration, it is possible to determine, byusing the prediction value determining information, whether a predictionvalue will be set to be a decoded offset or a predetermined value.

Application Examples

The above-described video decoding device 1 (1′) and video coding device2 (2′) may be mounted on and used for various apparatuses fortransmitting, receiving, recording, and playing back video images. Videoimages may be non-artificial video images captured by, for example, acamera, or artificial video images (including CG and GUI) generated by,for example, a computer.

The use of the above-described video decoding device 1 and video codingdevice 2 for transmitting and receiving video images will be describedbelow with reference to FIG. 45.

Part (a) of FIG. 45 is a block diagram illustrating the configuration ofa transmitting apparatus A on which the video coding device 2 ismounted. As shown in part (a) of FIG. 45, the transmitting apparatus Aincludes a coder A1 which codes video images so as to obtain coded data,a modulator A2 which modulates a carrier wave by using the coded dataobtained by the coder A1 so as to obtain a modulation signal, and atransmitter A3 which transmits the modulation signal obtained by themodulator A2. The above-described video coding device 2 is used as thiscoder A1.

As supply sources for inputting video images into the coder A1, thetransmitting apparatus A may also include a camera A4 which capturesvideo images, a recording medium A5 on which video images are recorded,an input terminal A6 for receiving video images from an external source,and an image processor A7 which generates or processes images. Althoughthe configuration of the transmitting apparatus A including all theseelements is shown in part (a) of FIG. 45, some elements may be omitted.

The recording medium A5 may be a recording medium on which video imageswhich are not coded or video images which are coded by a recordingcoding method, which is different from a transmitting coding method, arerecorded. If video images which are coded by a recording coding methodare recorded on the recording medium A5, a decoder (not shown) fordecoding coded data read from the recording medium A5 by using arecording coding method may be interposed between the recording mediumA5 and the coder A1.

Part (b) of FIG. 45 is a block diagram illustrating the configuration ofa receiving apparatus B on which the video decoder 1 is mounted. Asshown in part (b) of FIG. 45, the receiving apparatus B includes areceiver B1 which receives a modulation signal, a demodulator B2 whichdemodulates the modulation signal received by the receiver B1 so as toobtain coded data, and a decoder B3 which decodes the coded dataobtained by the demodulator B2 so as to obtain video images. Theabove-described video decoding device 1 is used as this decoder B3.

As destinations to which video images output from the decoder B3 aresupplied, the receiving apparatus B may also include a display B4 whichdisplays video images, a recording medium B5 for recording video imagesthereon, and an output terminal B6 for outputting video images to anexternal source. Although the configuration of the receiving apparatus Bincluding all these elements is shown in part (b) of FIG. 45, someelements may be omitted.

The recording medium B5 may be a recording medium for recording thereonvideo images which are not coded or video images which are coded by arecording coding method, which is different from a transmitting codingmethod. If the recording medium B5 is a recording medium for recordingvideo images which are coded by a recording coding method, a coder (notshown) for coding video images obtained from the decoder B3 by using arecording coding method may be interposed between the decoder B3 and therecording medium B5.

A transmission medium for transmitting a modulation signal may be awireless medium or a wired medium. A transmission mode in which amodulation signal is transmitted may be broadcasting (in this case, atransmission mode in which a transmission destination is not specifiedin advance) or communication (in this case, a transmission destinationis specified in advance). That is, transmission of a modulation signalmay be implemented by any one of radio broadcasting, cable broadcasting,radio communication, and wired communication.

For example, a broadcasting station (such as broadcasting equipment) anda receiving station (such as a television receiver) of terrestrialdigital broadcasting are respectively an example of the transmittingapparatus A which transmits a modulation signal via radio broadcastingand an example of the receiving apparatus B which receives a modulationsignal via radio broadcasting. A broadcasting station (such asbroadcasting equipment) and a receiving station (such as a televisionreceiver) of cable television broadcasting are respectively an exampleof the transmitting apparatus A which transmits a modulation signal viacable broadcasting and an example of the receiving apparatus B whichreceives a modulation signal via cable broadcasting.

A server (such as a workstation) and a client (such as a televisionreceiver, a personal computer, or a smartphone) using VOD (Video OnDemand) services or video hosting services on the Internet arerespectively an example of the transmitting apparatus A which transmitsa modulation signal via communication and the receiving apparatus Bwhich receives a modulation signal via communication (generally, in thecase of a LAN, a wired or wireless transmission medium is used, and inthe case of a WAN, a wired transmission medium is used). Examples of thepersonal computer are a desk-top PC, a laptop PC, and a tablet PC. Anexample of the smartphone is a multifunction mobile phone terminal.

A client using video hosting services has a function of coding videoimages captured by a camera and uploading the coded video images to aserver, as well as a function of decoding coded data downloaded from aserver and displaying the decoded data on a display. That is, a clientusing video hosting services serves as both of the transmittingapparatus A and the receiving apparatus B.

The use of the above-described video decoding device 1 and video codingdevice 2 for recording and playing back video images will be describedbelow with reference to FIG. 46.

Part (a) of FIG. 46 is a block diagram illustrating the configuration ofa recording apparatus C on which the video coding device 2 is mounted.As shown in part (a) of FIG. 46, the recording apparatus C includes acoder C1 which codes video images so as to obtain coded data, and awriter C2 which writes the coded data obtained by the coder C1 into arecording medium M. The above-described video coding device 2 is used asthis coder C1.

The recording medium M may be (1) a type integrated in the recordingapparatus C, such as an HDD (Hard Disk Drive) or an SSD (Solid StateDrive), (2) a type connected to the recording apparatus C, such as a SDmemory card or a USB (Universal Serial Bus) flash memory, or (3) a typeloaded in a drive (not shown) integrated in the recording apparatus C,such as a DVD (Digital Versatile Disc) or a BD (Blu-ray Disc: registeredtrademark).

As supply sources for inputting video images into the coder C1, therecording apparatus C may also include a camera C3 which captures videoimages, an input terminal C4 for receiving video images from an externalsource, a receiver C5 for receiving video images, and an image processorC6 which generates or processes images. Although the configuration ofthe recording apparatus C including all these elements is shown in part(a) of FIG. 46, some elements may be omitted.

The receiver C5 may be a receiver which receives video images which arenot coded or video images which are coded by a transmitting codingmethod, which is different from a recording coding method. If thereceiver C5 is a receiver which receives video images coded by atransmitting coding method, a transmitting decoder (not shown) fordecoding coded data coded by a transmitting coding method may beinterposed between the receiver C5 and the coder C1.

Examples of the recording apparatus C are a DVD recorder, a BD recorder,and an HD (Hard Disk) recorder (in this case, the input terminal C4 orthe receiver C5 is a main supply source for video images). Otherexamples of the recording apparatus C are a camcorder (in this case, thecamera C3 is a main supply source for video images), a personal computer(in this case, the receiver C5 or the image processor C6 is a mainsupply source for video images), and a smartphone (in this case, thecamera C3 or the receiver C5 is a main supply source for video images).

Part (b) of FIG. 46 is a block diagram illustrating the configuration ofa playback apparatus D on which the video decoding device 1 is mounted.As shown in part (b) of FIG. 46, the playback apparatus D includes areader D1 which reads coded data written into a recording medium M, anda decoder D2 which decodes the coded data read by the reader D1 so as toobtain video images. The above-described video decoding device 1 is usedas this decoder D2.

The recording medium M may be (1) a type integrated in the playbackapparatus D, such as an HDD or an SSD, (2) a type connected to theplayback apparatus D, such as a SD memory card or a USB flash memory, or(3) a type loaded in a drive (not shown) integrated in the playbackapparatus D, such as a DVD or a BD.

As destinations to which video images output from the decoder D2 aresupplied, the playback apparatus D may also include a display D3 whichdisplays video images, an output terminal D4 for outputting video imagesto an external source, and a transmitter D5 which transmits videoimages. Although the configuration of the playback apparatus D includingall these elements is shown in part (b) of FIG. 46, some elements may beomitted.

The transmitter D5 may be a transmitter which transmits video imageswhich are not coded or video images which are coded by a transmittingcoding method, which is different from a recording coding method. If thetransmitter D5 is a transmitter which transmits video images coded by atransmitting coding method, a coder (not shown) for coding video imagesby a transmitting coding method may be interposed between the decoder D2and the transmitter D5.

Examples of the playback apparatus D are a DVD player, a BD player, andan HDD player (in this case, the output terminal D4 to which atelevision receiver, for example, is connected is a main destination towhich video images are supplied). Other examples of the playbackapparatus D are a television receiver (in this case, the display D3 is amain destination to which video images are supplied), a desk-top PC (inthis case, the output terminal D4 or the transmitter D5 is a maindestination to which video images are supplied), a laptop or tablet PC(in this case, the display D3 or the transmitter D5 is a maindestination to which video images are supplied), a smartphone (in thiscase, the display D3 or the transmitter D5 is a main destination towhich video images are supplied), and digital signage (also called anelectronic bulletin board system, and in this case, the display D3 orthe transmitter D5 is a main destination to which video images aresupplied).

(Configuration Implemented by Software)

The individual blocks of the video decoding device 1(1′) and the videocoding device 2(2′), in particular, the variable-length code decoder 13,the motion-vector reconstructing unit 14, the inter-prediction imagegenerator 16, the intra-prediction image generator 17, theprediction-method determining unit 18, theinverse-quantize-and-inverse-transform unit 19, the deblocking filter41, the adaptive filter 50, the adaptive offset filter 60 (60′), thetransform-and-quantize unit 21, the variable-length code coder 22, theinverse-quantize-and-inverse-transform unit 23, the intra-predictionimage generator 25, the inter-prediction image generator 26, themotion-vector detector 27, the prediction-method controller 28, themotion-vector redundancy eliminating unit 29, the deblocking filter 33,the adaptive filter 70, and the adaptive offset filter 80 (80′) may beimplemented by hardware by using a logical circuit formed on anintegrated circuit (IC chip), or may be implemented by software by usinga CPU (central processing unit).

If the above-described elements are implemented by software, the videodecoding device 1 and the video coding device 2 each include a CPU whichexecutes commands of a control program which implements the individualfunctions, a ROM (read only memory) storing this program therein, a RAM(random access memory) loading this program, a storage device (recordingmedium), such as a memory, storing this program and various items ofdata therein, and so on. The object of the present invention may also beimplemented by supplying a recording medium on which program code (anexecution form program, an intermediate code program, and a sourceprogram) of the control program for the video decoding device 1 and thevideo coding device 2, which is software implementing theabove-described functions, is recorded in a computer readable manner, tothe video decoding device 1 and the video coding device 2, and byreading and executing the program code recorded on the recording mediumby a computer (or a CPU or an MPU (micro processing unit)) of each ofthe video decoding device 1 and the video coding device 2.

As the above-described recording medium, for example, a tape type, suchas magnetic tape or cassette tape, a disk type including a magneticdisk, such as a floppy (registered trademark) disk or a hard disk, andan optical disc, such as a CD-ROM (compact disc read-only memory), an MOdisc (magneto-optical disc), an MD (Mini Disc), a DVD (digital versatiledisc), or a CD-R (CD recordable), a card type, such as an IC card(including a memory card) or an optical card, a semiconductor memorytype, such as a mask ROM, an EPROM (erasable programmable read-onlymemory), an EEPROM (electrically erasable and programmable read-onlymemory) (registered trademark), or a flash ROM, or a logical circuittype, such as a PLD (Programmable logic device) or an FPGA (FieldProgrammable Gate Array), may be used.

The video decoding device 1 and the video coding device 2 may beconnected to a communication network and the above-described programcode may be supplied via the communication network. This communicationnetwork is not particularly restricted as long as it is capable oftransmitting the program code. For example, the Internet, an intranet,an extranet, a LAN (local area network), ISDN (integrated servicesdigital network), a VAN (value-added network), a CATV (community antennatelevision/cable television) communication network, a VPN (virtualprivate network), a public switched telephone network, a mobilecommunication network, a satellite communication work, etc. may be used.Additionally, a transmission medium forming this communication networkis not restricted to a specific configuration or a specific type as longas it is capable of transmitting the program code. For example, a wiredtransmission medium, such as IEEE (institute of electrical andelectronic engineers) 1394, USB, power line communication, a cable TVline, a telephone line, or an ADSL (asymmetric digital subscriber loop)circuit, or a wireless transmission medium, such as infrared, forexample, IrDA (infrared data association) or a remote controller,Bluetooth (registered trademark), IEEE802.11 radio, HDR (high datarate), NFC (Near Field Communication), DLNA (Digital Living NetworkAlliance), a mobile phone network, a satellite circuit, or a terrestrialdigital network, may be used. In the present invention, theabove-described program code may also be implemented in the form of acomputer data signal embedded in a carrier wave through digitaltransmission.

The present invention is not restricted to the above-describedembodiments, and various modifications may be made within the scope ofthe claims. Embodiments obtained by combining technical means disclosedin the different embodiments in an appropriate manner are alsoencompassed in the technical scope of the present invention.

INDUSTRIAL APPLICABILITY

The present invention is suitably applicable to an image filter whichperforms offset filtering on image data. The invention is also suitablyapplicable to a decoding device which decodes coded data and a codingdevice which codes coded data.

REFERENCE SIGNS LIST

-   -   1 video decoding device (decoding device)    -   41 deblocking filter    -   50 adaptive filter    -   60, 60′ adaptive offset filter (image filtering device)    -   611, 611′ offset information decoding section (offset decoding        device, determining means, offset attribute setting means,        offset decoding means, offset residual decoding means,        prediction value determining means)    -   612 QAOU structure decoding section    -   613 offset attribute setting section (offset attribute setting        means)    -   621 offset information storage section    -   622 QAOU controller    -   623 offset-type determining section    -   624 classifying section (calculating means, bit shift means,        classifying means)    -   625 offset determining section (offset reverse shift means)    -   626 offset adder (offset means)    -   2 video coding device (coding device)    -   33 deblocking filter    -   70 adaptive filter    -   80, 80′ adaptive offset filter (image filtering device)    -   811 offset calculator    -   812 offset clipping section    -   813 offset information selector    -   814 offset residual determining section    -   815 offset attribute setting section (offset attribute setting        means)    -   816 offset shifter (offset shift means)

The invention claimed is:
 1. An image filtering device, comprising: anoffset attribute setting unit, configured to set an upmost limitation ofan offset value range in accordance with a bit depth of pixel values ofpixels forming an input image, wherein the bit depth of pixel values ofpixels forming the input image is obtained from coded data; an offsetdecoding unit, configured to decode an offset value which is restrictedto the set offset value range, wherein an offset bit depth of the offsetvalue is equal to the bit depth of the pixel values in a case in whichthe bit depth of the pixel values is ten or smaller, or the offset bitdepth of the offset value is equal to ten in a case in which the bitdepth of the pixel values is eleven or greater; wherein the offsetattribute setting unit is configured to set a maximum bit lengthrepresenting the upmost limitation of the offset value range to be (theoffset bit depth−K) or smaller, wherein the upmost limitation of theoffset value range is determined to be(2^((the offset bit depth-K−1))−1), and K is an integer greater than 0;an offset-type determining unit, configured to determine, among firstand second offset types, an offset type to which a subject unit areaincluding the pixel forming the input image belongs; and a filteringunit, configured to left-shift the offset value according to a shiftvalue, and add left-shifted offset value to a pixel value of a pixelincluded in the input image which is constituted by a plurality of unitareas, wherein the shift value is equal to (bit depth of the pixelvalue-offset bit depth of the offset value).
 2. The image filteringdevice according to claim 1, wherein a value of K is equal to
 4. 3. Animage filtering method, comprising: setting, by a Sample Adaptive Offset(SAO) filter, an upmost limitation of an offset value range inaccordance with a bit depth of pixel values of pixels forming an inputimage, wherein the bit depth of the pixel values of pixels forming theinput image is obtained from coded data; decoding, by a decoder, anoffset value which is restricted to the set offset value range, whereinan offset bit depth of the offset value is equal to the bit depth of thepixel values in a case in which the bit depth of the pixel values is tenor smaller, or the offset bit depth of the offset value is ten in a casein which the bit depth of the pixel values is eleven or greater, and,wherein a maximum bit length representing the upmost limitation of theoffset value range is set by the SAO filter to be (the offset bitdepth−K) or smaller, wherein the upmost limitation of the offset valuerange is determined to be (2^((the offset bit depth-K−1))−1), and K isan integer greater than 0; determining, among first and second offsettypes, an offset type to which a subject unit area including the pixelforming the input image belongs; left-shifting the offset value by theSAO filter according to a shift value and adding by the SAO filter theleft-shifted offset value to a pixel value of a pixel included in theinput image which is constituted by a plurality of unit areas, whereinthe shift value is equal to (bit depth of the pixel value minus offsetbit depth of the offset value).
 4. The image filtering method accordingto claim 3, wherein a value of K is equal to
 4. 5. An image filteringdevice comprising: a non-transitory memory having processor-executableinstructions stored thereon; and a processor in communication with thenon-transitory memory, the processor being configured to execute theprocessor-executable instructions stored in the non-transitory memoryfor: setting an upmost limitation of an offset value range in accordancewith a bit depth of pixel values of pixels forming an input image,wherein the a bit depth of the pixel values of the pixels forming theinput image is obtained from coded data; decoding an offset value whichis restricted to the set offset value range, wherein an offset bit depthof the offset value is equal to the bit depth of the pixel values in acase in which the bit depth of the pixel values is ten or smaller, orthe offset bit depth of the offset value is equal to ten in a case inwhich the bit depth of the pixel values is eleven or greater, and,wherein a maximum bit length representing the upmost limitation of theoffset value range is set by the SAO filter to be (the offset bitdepth−K) or smaller, wherein the upmost limitation of the offset valuerange is determined to be (2^((the offset bit depth-K−1))−1), and K isan integer greater than 0; determining, among first and second offsettypes, an offset type to which a subject unit area including the pixelforming the input image belongs; left-shifting the offset valueaccording to a shift value, and adding left-shifted offset value to apixel value of a pixel included in the input image which is constitutedby a plurality of unit areas, wherein the shift value is equal to (bitdepth of the pixel value minus offset bit depth of the offset value). 6.The image filtering device according to claim 5, wherein a value of K isequal to
 4. 7. An image filtering device, comprising: an offsetattribute setting unit, configured to set an upmost limitation of anoffset value range in accordance with a bit depth of pixel values ofpixels forming an input image, wherein the bit depth of pixel values ofpixels forming the input image is obtained from coded data, and theinput image is decoded from the coded data; an offset decoding unit,configured to decode an offset value which is restricted to the setoffset value range; an offset-type determining unit, configured todetermine, among first and second offset types, an offset type to whicha subject unit area including the pixel forming the input image belongs;a filtering unit, configured to add the offset value associated with theoffset type to a pixel value of a pixel included in the input imagewhich is constituted by a plurality of unit areas; wherein an offset bitdepth of the offset value is equal to the bit depth of the pixel valuesin a case in which the bit depth of the pixel values is ten or smaller,or the offset bit depth of the offset value is equal to ten in a case inwhich the bit depth of the pixel values is eleven or greater; whereinthe offset attribute setting unit is configured to set a maximum bitlength representing the upmost limitation of the offset value range tobe (the offset bit depth−K) or smaller, wherein the upmost limitation ofthe offset value range is determined to be(2^((the offset bit depth-K−1))−1), and K is an integer greater than 0.8. The image filtering device according to claim 7, wherein a value of Kis equal to
 4. 9. An image filtering method, comprising: setting, by aSample Adaptive Offset (SAO) filter, an upmost limitation of an offsetvalue range in accordance with a bit depth of pixel values of pixelsforming an input image, wherein the bit depth of the pixel values ofpixels forming the input image is obtained from coded data, and theinput image is decoded from the coded data; decoding, by a decoder, anoffset value which is restricted to the set offset value range;determining, among first and second offset types, an offset type towhich a subject unit area including the pixel forming the input imagebelongs; adding, by the SAO filter, the offset value associated with theoffset type to a pixel value of a pixel included in the input imagewhich is constituted by a plurality of unit areas; wherein an offset bitdepth of the offset value is equal to the bit depth of the pixel valuesin a case in which the bit depth of the pixel values is ten or smaller,or the offset bit depth of the offset value is equal to ten in a case inwhich the bit depth of the pixel values is eleven or greater; andwherein the maximum bit length representing the upmost limitation of theoffset value range is set by the SAO filter to be (the offset bitdepth−K) or smaller, wherein the upmost limitation of the offset valuerange is determined to be (2^((the offset bit depth-K−1))−1), and K isan integer greater than
 0. 10. The image filtering method according toclaim 9, wherein a value of K is equal to
 4. 11. An image filteringdevice comprising: a non-transitory memory having processor-executableinstructions stored thereon; and a processor in communication with thenon-transitory memory, the processor being configured to execute theprocessor-executable instructions stored in the non-transitory memoryfor: setting an upmost limitation of an offset value range in accordancewith a bit depth of pixel values of pixels forming an input image,wherein the a bit depth of the pixel values of the pixels forming theinput image is obtained from coded data, and the input image is decodedfrom the coded data; decoding an offset value which is restricted to theset offset value range; determining, among first and second offsettypes, an offset type to which a subject unit area including the pixelforming the input image belongs; adding the offset value to a pixelvalue associated with the offset type of a pixel included in the inputimage which is constituted by a plurality of unit areas; wherein anoffset bit depth of the offset value is equal to the bit depth of thepixel values in a case in which the bit depth of the pixel values is tenor smaller, or the offset bit depth of the offset value is equal to tenin a case in which the bit depth of the pixel values is eleven orgreater; and wherein the maximum bit length representing the upmostlimitation of the offset value range is set to be (the offset bitdepth−K) or smaller, wherein the upmost limitation of the offset valuerange is determined to be (2^((the offset bit depth-K−1))−1), and K isan integer greater than
 0. 12. The image filtering device according toclaim 11, wherein a value of K is equal to 4.